Play back apparatus, playback method and program for playing back 3d video

ABSTRACT

The playback apparatus realizes stereoscopic viewing by overlaying planar or stereoscopic graphics over stereoscopic video in a way that reduces eye strain using following method in abstract: A graphics plane holds therein data composed of graphics data. A shift engine shifts, in a case when a composition unit composites the graphics data with a left-view video frame, coordinates of each of the pixels is shifted in a first horizontal direction, and in a case when the composition unit composites the graphics data with a right-view video frame, coordinates of each of the pixels is shifted in a second horizontal direction that is opposite to the first direction.

CROSS REFERENCE TO RELATED APPLICATION

The present application is a continuation application of U.S. patentapplication Ser. No. 12/508,207, filed on Jul. 23, 2009, which claimspriority to U.S. Provisional Pat. Appl. No. 61/111,045, filed on Nov. 4,2008. The disclosure of each of these documents, including thespecification, drawings, and claims, is incorporated herein by referencein its entirety.

The disclosure of each of Japanese Pat. Appl. No. 2009-101846, filed onApr. 20, 2009, and Japanese Pat. Appl. No. 2008-190525, filed on Jul.24, 2008, including the specification, drawings, and claims, isincorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention belongs to a technical field of a graphicscomposition technique.

(2) Description of the Related Art

The graphics composition technique is a technique of compositinggraphics such as subtitles or a GUI, to each frame of video whichconstructs a video stream, then displaying the result. As a technicaltrend of display device, displays that can display stereoscopic video aswell as planar video are becoming popular. Various methods ofstereoscopic viewing are adopted in the stereoscopic display devices,however, nearly all of the methods use the basic principle of displayingeach of left eye and a right eye a different image to createstereoscopic effect, using binocular disparity.

In order to allow the viewers to view the stereoscopic video at the sameframe rate as normal planar video, response performance twice as high asresponse performance needed for the normal planar video is necessary.This means, for example, that it is necessary to switch-among at least120 frames per second in order to display video consisting of 60 framesper second.

Accordingly, the video stream to be displayed needs to be encoded at 120frames per second. A stereoscopic effect may also be obtained withoutincreasing the frame rate, by using a side-by-side method as describedin a Non-Patent Document 1 or checker pattern method as described inPatent Document 2.

There is also, a technique known to generate stereoscopic images, thatextracts information indicating the number of objects from a 2D video,then creating the number of layers corresponding to the number ofobjects so stereoscopic images can be generated by changing the depth ofeach of the layers as disclosed in a Patent Document 3.

PRIOR ART DOCUMENTS

[Patent Documents]

-   [Patent Document 1] International Publication No. 2005/119675    pamphlet-   [Patent Document 2] US Patent Application Publication No.    2008-0036854-   [Patent Document 3] US Patent Application Publication No.    2002-0118275

[Non-Patent Document]

-   [Non-Patent Document 1] FOUNDATIONS OF THE STEREOSCOPIC CINEMA A    STUDY IN DEPTH (by Lenny LIPTON)

SUMMARY OF THE INVENTION The Problems to be Solved by the Invention

As the situation now stands, application of a method of allowing viewersto enjoy viewing video streams stereoscopically is mainly engaged attheaters and the like. However, it is expected that it will be commonfor the viewers to enjoy viewing the stereoscopic video streams with useof playback apparatuses for household use in the future.

In package media such as BD-ROMs and DVD-videos, subtitles are notpre-embedded as part of video like the movies. In package media,plurality of subtitle data corresponding to plurality of languages isrecorded separately from a video stream. This is because it is necessaryto composite appropriate subtitle data with a video in accordance with alanguage setting in a playback apparatus, and to display data obtainedas a result of the composition.

Also, according to graphics for realizing the GUI, a plurality of piecesof graphics data is recorded separately from the video stream. This isalso because it is necessary to composite appropriate graphics data witha video in accordance with a language setting in a playback apparatus,and to display data obtained as a result of the composition. In thiscase, it is desirable that both of data for a left view (left-view data)and data for a right view (right-view data) are prepared for each ofgraphics showing subtitles, and graphics showing the GUI. However, sinceBD-ROMs have limited capacity, it may not be possible to prepare boththe left-view data and the right-view data for each of the video and thegraphics.

In a case where only one of the left-view data or the right-view datacan be prepared for the graphics, the graphics will appear to be flatwhile the video corresponding to the stereoscopic video stream can beviewed stereoscopically. Suppose that the planar graphics is compositedwith the stereoscopic video stream without the consideration of spacingcaused by the stereoscopic effect of displaying separate left-view videoand right-view video data. In such case, a flat graphics will appearwithin a stereoscopic video as a result of composition. As a result, thegraphics will appear to be buried into the stereoscopic video, which inturn will cause displeasure to the viewers. This is caused by the eyesnot being able to adjust to an unnatural stereoscopic composition notseen in real life, leading to decrease in realistic sensation.

The present invention has an objective to provide a playback apparatusthat is capable of executing stereoscopic playback by performingcomposition of the video and the graphics so as to prevent viewers fromfeeling uncomfortable despite when only graphics for one eye, eithergraphics for left view (left-view graphics) or graphics for right view(right-view graphics), is recorded on a recording medium.

Means to Solve the Problems

In order to solve the above-stated problem, the present invention is aplayback apparatus that executes stereoscopic playback, the playbackapparatus comprising: a video decoder operable to decode a video streamto obtain video frames; a video plane that holds therein the videoframes; a graphics plane that holds therein graphics data, the graphicsdata having a resolution of a predetermined number of horizontal andvertical pixels; a composition unit operable to composite the graphicsdata within the graphics plane with one of the video frames; and a shiftengine operable to perform plane shifting of the graphics plane, whereineach of the video frames in the video plane is outputted as a right-viewvideo frame or a left-view video frame, and the stated plane shifting ofthe graphics plane is defined as; prior to outputting the left-viewvideo frame, shifting each of the pixels of the graphics data within thegraphics plane in a right or a left direction, and prior to outputtingthe right-view video frame, shifting each of the pixels of the graphicsdata within the graphics plane in an opposite direction from left view,and then giving the resulted shifted graphics data within the graphicsplane to the composition unit for the composition.

Effects of the Invention

By shifting the coordinates of each of the pixel data pieces held in thegraphics plane (hereinafter, referred to as “plane shifting of thegraphics plane), it will become possible to adjust the depth position ofthe graphics to be displayed. The stated depth position meaning howclose or far the objects on the plane will appear to the user inaccordance to the display depth. Therefore, even in a case wheresubtitles and GUI appear to be buried in the stereoscopic video becauseboth of the right-view graphics (right-view interactive graphics) andthe left-view graphics (left-view interactive graphics) are not providedin a recording medium, a more natural stereoscopic composition (graphicsdepth position is closer to the viewer than the video depth position)can be achieved. Since the depth position of the subtitles and GUI arechanged by such an adjustment, it is not necessary to prepare both theleft-view graphics and the right-view graphics and only prepare 2Dgraphics for displaying the subtitles and the GUI stereoscopically oradding the depths thereto. Therefore, even in a case where a BD-ROMwhose capacity is limited is a target for the playback, it is possibleto provide the user with preferable stereoscopic view.

Also for authoring purposes, it will become possible to re-use thegraphics for 2D playback, thus will be able to skip the conversionprocess of stereoscopic graphics (create a second view), resulting in areduction of labor cost for generating stereoscopic contents.

Even when the depths of the graphics showing the subtitles and GUI arenot appropriate, it is possible to change the depth of the graphics toan appropriate depth in accordance with the depth of video to beoutputted, by performing the above-stated shifting. Therefore, thegraphics to which the depth is added appears to be natural, and therewill be no difference in how the video and the graphics showing thesubtitles and GUI appear stereoscopically. Therefore, stress caused tothe naked eyes of the viewers is reduced.

Also, since it is not necessary to load the left-view graphics and theright-view graphics in a memory separately, the memory capacity of theplayback apparatus will not be consumed even when an image obtained bycompositing the video with the graphics is displayed stereoscopically.

With the above-stated structure, the stereoscopic video playbackapparatus pertaining to the present invention can stereoscopicallydisplay the subtitles and the GUI on the video even if both of theleft-view graphics and right-view graphics are not prepared.

BRIEF DESCRIPTION OF THE DRAWINGS

These and the other objects, advantages and features of the inventionwill become apparent from the following description thereof taken inconjunction with the accompanying drawings which illustrate a specificembodiment of the invention. In the drawings:

FIG. 1 shows a usage pattern of a recording medium and a playbackapparatus 200;

FIG. 2 shows an internal structure of a BD-ROM 100;

FIG. 3 shows an internal structure of a BD-J object;

FIG. 4 shows an internal structure of the playback apparatus 200;

FIG. 5 shows detailed structures of a front end unit 101, a system LSI,a nonvolatile memory 105, a memory device 13, a back end unit and a hostmicrocomputer 106 in detail;

FIG. 6 shows switching between a 2D display mode and a 3D display mode;

FIG. 7 shows an example of composition processing when a stereo mode ofeach plane is ON, and an example of composition processing when a stereomode of each plane is OFF;

FIG. 8 shows how data in a background plane, data in a video plane 6,data in an image plane 8 and data in an interactive graphics plane 10are composited when stereo modes of all the planes are ON;

FIG. 9 shows how data in a background plane, data in a video plane 6 anddata in an interactive graphics plane 10 are composited when stereomodes of all the planes are OFF;

FIG. 10 shows a composition result for each plane;

FIG. 11 shows an example where an image that is outputted when thestereo modes of all the planes are ON is viewed on a 3D display;

FIG. 12 shows an example of how the stereoscopic video appears when theviewer sees, through shutter glasses 500, an image that is outputtedwhen the stereo mode of the video plane 6 is ON, and the stereo modes ofother planes are OFF is viewed;

FIG. 13A and FIG. 13B show an image in a left-view graphics planeshifted in a right direction, and an image in a right-view graphicsplane shifted in a left direction, respectively;

FIG. 14A and FIG. 14B show an internal structure of the image plane 8;

FIG. 15A, FIG. 15B and FIG. 15C show pixel data pieces in a foregroundarea and pixel data pieces in a background area after a plane shiftengine 19 shifts coordinates of each pixel data piece in the rightdirection, and shifts the coordinates of each pixel data piece in theleft direction; Each of FIG. 16A and FIG. 16B shows an internalstructure of the interactive graphics plane 10;

FIG. 17A, FIG. 17B and FIG. 17C show pixel data pieces in a foregroundarea and pixel data pieces in a background area after the plane shiftengine 19 shifts the coordinates of each pixel data piece in the rightdirection, and shifts the coordinates of each pixel data piece in theleft direction;

FIG. 18A, FIG. 18B and FIG. 18C show processing procedures for shiftingthe coordinates of each pixel data piece held in the image plane 8;

FIG. 19A, FIG. 19B and FIG. 19C show processing procedures for shiftingthe coordinates of each pixel data piece held in the interactivegraphics plane 10;

FIG. 20A and FIG. 20B show how depth of subtitles changes by a negativeoffset and a positive offset;

FIG. 21 shows pixel data pieces in the graphics plane;

Each of FIG. 22A and FIG. 22B shows what is held in the graphics planeafter the plane shift engine 19 shifts the coordinates of each of thepixel data pieces;

FIG. 23 shows an internal structure of a BD-J platform unit;

FIG. 24 shows what is stored in a display mode storage unit 29;

FIG. 25 shows, in a table, changes in display modes when a title ischanged;

FIG. 26 shows switching between display modes when a playlist beingplayed back is changed in each title;

FIG. 27A and FIG. 27B show API used for setting by the display modestorage unit 29;

FIG. 28 shows an internal structure of an offset setting unit;

FIG. 29 shows a flowchart showing processing procedures for display modesetting when a title is changed;

FIG. 30 shows a flowchart showing processing procedures for display modesetting in each title;

FIG. 31A, FIG. 31B and FIG. 31C describe principles of how a planerimage appears to be in a position closer to a viewer than a position ofa display screen;

FIG. 32 shows a flowchart showing main procedures for playing back aplaylist;

FIG. 33 shows a flowchart showing playback procedures based on playiteminformation;

FIG. 34 shows a flowchart showing left-view processing of a 3D stream inthe 3D display mode;

FIG. 35 shows a flowchart showing right-view processing for a 3D streamin the 3D display mode;

FIG. 36 shows a flowchart showing processing procedures for a 2D AVstream in the 2D display mode; and

FIG. 37 shows a flowchart showing processing procedures when the BD-Japplication gives an offset update instruction;

FIG. 38 shows a structure of a composition unit 15;

FIG. 39A, FIG. 38B and FIG. 38C show processes for executing a shiftingof each line data in the right direction;

FIG. 40A, FIG. 40B and FIG. 40C show processes for executing shifting ofcoordinates of each pixel data piece in each line data in the plane inthe left direction;

FIG. 41 shows a flowchart showing processing procedures for storing datain a line memory; and

FIG. 42A, FIG. 42B and FIG. 42C describe principles of how a planarimage appears to be in a position more distant from the viewer than aposition of the display screen.

DESCRIPTION OF PREFERRED EMBODIMENT

The following describes the embodiments of a recording medium and aplayback apparatus 200 having the above-stated Means to Solve theProblems with reference to the drawings.

FIG. 1 shows a usage pattern of the recording medium and the playbackapparatus 200. As shown in FIG. 1, a BD-ROM 100 (taken as an example ofrecording medium) and a playback apparatus 200 compose a home theatersystem together with a remote control 300, TV 400 and liquid crystalshutter glasses 500, and are used by a user.

The BD-ROM 100 provides the above-stated home theater system with amovie, for example.

The playback apparatus 200 is connected to the TV 400, and plays backthe BD-ROM 100.

The remote control 300 is a device that receives an operation for thehierarchized GUI from the user. In order to receive such an operation,the remote control 100 includes: a menu key for calling menus composingthe GUI; an arrow key for moving a focus of the GUI parts composing eachmenu; a decision key for performing determined operation on the GUIparts composing each menu; a return key for returning to higher orderhierarchized menus; and a numerical key.

The TV 400 provides the user with a dialogical operational environmentby displaying the playback picture of the movie, a menu and the like.

The liquid crystal shutter glasses 500 are composed of crystal liquidshutters and a control unit, and realize stereoscopic view with use ofbinocular disparity of the viewer's eyes. Lenses having a feature thatthe transmittance of light is changed by changing applied voltage areused for the crystal liquid shutters of the liquid crystal shutterglasses 500. The control unit of the liquid crystal shutter glasses 500receives a SYNC signal for switching between a right-view image and aleft-eye image that are transmitted from the playback apparatus 200, andswitches between a first state and a second state in accordance withthis SYNC signal.

In the first state, the control unit adjusts the applied voltage suchthat light does not transmit through a liquid crystal lens correspondingto the right view, and adjust the applied voltage such that lighttransmits through a liquid crystal lens corresponding to the left view.In such a state, only the left-view image is viewed.

In the second state, the control unit adjusts the applied voltage suchthat the liquid crystal lens corresponding to the right view transmitslight, and adjusts the applied voltage such that the liquid crystal lenscorresponding to the left view does not transmit light. In, such astate, the crystal liquid shutters provide view of the right-view image.

Generally, the right-view image and left-view image look a littledifferent due to differences between angles. With use of such adifference, the user can recognize an image as a stereoscopic image.Thus, the user confuses a planar display with a stereoscopic display bysynchronizing timing of switching between the above-stated first stateand second state with timing of switching between the right-view imageand the left-view image. Next, a description is given of a time intervalin displaying right-view video and left-view video.

Specifically, there is a difference between the right-view image and theleft-view image that corresponds to binocular disparity of the user in aplanar image. By displaying theses images while switching the images ata short time interval, the images appear as if the images are displayedstereoscopically.

The short time interval may be a time period just enough to confuse theuser on planar images with stereoscopic images when the switching andthe displaying are performed as stated in the above.

This concludes the description of the home theater system.

The following describes a recording medium to be played back by theplayback apparatus 200. The playback apparatus 200 plays back the BD-ROM100. FIG. 2 shows the internal structure of the BD-ROM 100.

The BD-ROM is shown in a fourth tier from the top in the present figure,and a track on the BD-ROM is shown in a third tier. Although the trackis usually formed to a spiral manner from an inner circumference to anouter circumference, the track is drawn in a laterally-expanded mannerin the present figure. This track consists of a read-in area, a volumearea and a read-out area. Also, in the read-in area exists a specialarea called BCA (Burst Cutting Area) that can be read only by a drive.Since this area cannot be read by an application, this area is oftenused in copyright protection technology.

The volume area in the present figure has a layer model having aphysical layer, a file system layer and an application layer.Application data such as image data starting with file systeminformation (volume) is stored in the volume area. The file system is.UDF, ISO9660 or the like. In the file system, it is possible to readlogic data recorded in the same manner as a normal PC, with use of adirectory or a file structure. Also, a file name or a directory nameconsisting of 255 words can be read. A top tier of FIG. 2 shows anapplication layer format (application format) of the BD-ROM expressedusing a directory structure. As shown in the first tier, in the BD-ROM,a CERTIFICATE directory and a BDMV directory exists below the Rootdirectory.

Below the CERTIFICATE directory, a file of a root certificate(app.discroot.certificate) of a disc exists. Thisapp.discroot.certificate is a digital certificate used in a process thatchecks whether an application has been tampered, or identifies theapplication (hereinafter, signature verification) when executing aprogram of a JAVA™ application that performs dynamic scenario controlusing a JAVA™ virtual machine.

The BDMV directory is a directory in which data such as AV content andmanagement information used in the BD-ROM are recorded. Six directoriescalled “PLAYLIST directory”, “CLIPINF directory”, “STREAM directory”,“BDJO directory”, “JAR directory” and “META directory” exist below theBDMV directory. Also, two types of files (i.e. INDEX.BDMV andMovieObject.bdmv) are arrayed.

The STREAM directory is a directory storing a file which is a so-calledtransport stream body. A file (000001.m2ts) to which an extension “m2ts”is given exists in the STREAM directory.

A file (000001.mpls) to which an extension “mpls” is given exists in thePLAYLIST directory.

A file (000001.clpi) to which an extension “clpi” is given exists in theCLIPINF directory.

A file (XXXXX.bdjo) to which an extension “bdjo” is given exists in theBDJO directory.

A file (YYYYY.jar) to which an extension “jar” is given exists in theJAR directory.

An XML file (ZZZZZ.xml) exists in the META directory.

The following describes these files.

Firstly, a description is given of the file to which the extension“m2ts” is given. The file to which the extension “m2ts” is given is adigital AV stream in the MPEG-TS (Transport Stream) method, and isacquired by multiplexing a video stream, one or more audio streams, agraphics stream, a text subtitle stream and the like. The video streamrepresents moving part of the movie, and the audio stream representsaudio part of the movie. In the case of the 3D stream, both of left-eyedata and right-eye data may be included in m2ts, or m2ts may be preparedseparately for each of the left-eye data and the right-eye data. It ispreferable to use a codec (e.g. MPEG-4 AVC MVC) in which a left-viewstream and a right-view stream refer to one another in order to savedisc capacity used for streams. Video streams compressed and encodedwith use of such a codec are called MVC video streams.

There are 2 types of MVC video streams, base view video stream andenhanced view video stream. The base view video stream is a video streamcomposing either the left view or the right view video that realizesplanar view display. Meanwhile, the “enhanced view video stream” is thevideo stream that is used with the base view to construct stereoscopicvideo, composing the left or right view video that is not composed inthe base view video stream. Picture data pieces composing the enhancedview video stream are compressed and encoded based on framecorrelativity with picture data pieces composing the base view stream.

The file to which the extension “mpls” is given is a file storingPlayList (PL) information. The PL information defines a playlistreferring to the AVClip.

In the present embodiment, it is possible to determine whether streamsto be played back include a 3D video stream, based on a structuralformat of the PlayList (PL) stored on the BD-ROM.

The PlayList information includes MainPath information, Subpathinformation and PlayListMark information.

1) The MainPath information defines a logic playback section by definingat least one pair of a time point (In_Time) and a time point (Out_Time)on a playback time axis of the AV stream. The MainPath information has astream number table (STN_table) that stipulates which elementary streamsthat have been multiplexed into AV stream are permitted to be playedback and are not permitted to be played back.

2) The PlayListMark information shows specification of a time pointcorresponding to a chapter in a part of the AV stream specified by thepair of the In_Time information and the Out_Time information.

3) The Subpath information is composed of at least one piece ofSubPlayItem information. The SubPlayItem information includesinformation on specification of an elementary stream to be played backin synchronization with the AV stream, and includes a pair of In_Timeinformation and Out_Time information on the playback time axis of theelementary stream. The Java™ application for controlling playbackinstructs a Java™ virtual machine to generate a JMF (Java Media Framework) player instance that plays back this PlayList information. Thisstarts the playback of the AV stream. The JMF player instance is actualdata generated on a heap memory of the virtual machine based on the JMFplayer class.

Hereinafter, a stream including only a stream for 2D playback isreferred to as a “2D stream”, and a stream including both a 2D and 3Dstreams is referred to as a “3D stream”.

Furthermore, according to a term definition, a 2D playlist is a playlistincluding only a stream for the 2D playback while a 3D playlist includesa stream for 3D viewing in addition to the 2D stream.

The file to which the extension “clpi” is given is Clip informationwhich is in one to one correspondence with AVclip information. Since theClip information is management information, the Clip information has anEP_map showing an encoding format of the stream in the AVClip, a framerate, a bit rate, information on resolution and the like, and a startingpoint of GOPs. The Clip information and PL information are classified as“static scenario”.

The following describes the file to which the extension “BD-J object” isgiven. The file to which the extension “BD-J object” is given is a filestoring a BD-J object. The BD-J object is information that defines atitle by association an AVClip string defined by the PlayListinformation with an application.

The entity of the Java™ application corresponds to a Java™ archive file(YYYYY.jar) stored in the JAR directory under the BDMV directory in FIG.2.

The application is a Java™ application, and is composed of one or morexlet programs loaded in the heap area (also called “work memory”) of thevirtual machine. The application is composed of the xlet program loadedin the work memory and data.

In the meta file (ZZZZZ.xml) included in the META directory is storedvarious information pieces relating to the movie in the disc. Examplesof information pieces stored in the meta file are a name of the disc andan image of the disc, information on who has created the disc, and atitle name for each title. This concludes the description of the BD-RO100. The meta file is not a prerequisite, and some BD-ROMs do notinclude this meta file. This concludes the description of the BD-ROM.

The following describes the BD-J object. FIG. 3 shows the internalstructure of the BD-J object. As shown in FIG. 3, the BD-J object iscomposed of an “application management table”, a “GUI management table”and a “playlist management table”.

The following describes these elements.

The “application management table” is a table for causing the playbackapparatus 200 to perform application signaling that runs the title as alife cycle. A lead line bj1 shows the internal structure of theapplication management table in closeup. As shown in this leadline, theapplication management table includes an “application identifier” and a“control code” that specify an application to be operated when a titlecorresponding to the BD-J object becomes a current title. When thecontrol code is set to AutoRun, the control code shows that thisapplication is automatically started after the application is loaded onthe heap memory. When the control code is set to Present, the controlcode waits for a call from another application, and shows whether theapplication should be operated after the application is loaded on theheap memory.

The “GUI management table” is a table used for GUI related operations bythe application. More specifically, resolution, font data used, maskinginformation when GUI for executing the menu call, or a title call isinstructed by the user, is included. A lead line bj2 shows the internalstructure of the GUI management table in closeup. As shown by this leadline bj2, the GUI management table may be set to one ofHD3D_(—)1920×1080, HD3D_(—)1280×720, HD_(—)1920×1080, HD_(—)1280×720,QHD960×540, SD, SD_(—)50HZ_(—)720_(—)576 and SD_(—)60HZ_(—)720_(—)480.

A lead line bj3 shows the internal structure of the playlist managementtable in closeup. The playlist management table includes info nation onspecification of the playlist to be operated by default when a titlecorresponding to the BD-J object becomes a current title. A lead linebj4 shows the internal structure of a auto start playlist in closeup. Asshown by the lead line bj4, 3D playlist 1920*1080, 3D playlist 1280*720,2D playlist 1920*1080, 2D playlist1 1280*720, 2D playlist 720*576 and 2Dplaylist 720*480 may be specified as information specifying the autostart playlist.

FIG. 4 shows the internal structure of the playback apparatus 200. Afront end unit 101, a system LSI 102, a memory device 103, a back endunit 104, a nonvolatile memory 105, a host microcomputer 106 and anetwork I/F 107 mainly compose a playback apparatus 200 in FIG. 4.

The front end unit 101 is a data input source. In a figure described inthe following, the front end unit 101 includes a BD drive 110 and alocal storage 111, for example.

The system LSI 102 is composed of logic elements, and is a core part ofthe playback apparatus 200. This system LSI includes therein at least ademultiplexer 4, video decoders 5 a and 5 b, image decoders 7 a and 7 b,an audio decoder 9, a playback state/setting register (PSR: PlayerStatus/Setting Register) set 12, a playback control engine 14, acomposition unit 15, a plane shift engine 19 and an offset setting unit20.

The memory device 103 is composed of arrays of memory devices such as anSDRAM.

The back end unit 104 is a connection interface that connects internalparts of the playback apparatus 200 with other devices.

The nonvolatile memory 105 is a readable and writable recording medium,and is a medium that can hold recorded contents without needing powersource supply. The nonvolatile memory 105 is used for backup ofinformation on a display mode stored in a display mode storage unit 24(described in the following). A flash memory, an FeRAM or the like maybe used as this nonvolatile memory 105.

The host microcomputer 106 is a core part of the playback apparatus 200,and is composed of an MPU, a ROM and a RAM. The host microcomputer 106includes a BD-J platform 22 and a command interpreter 25 from among theelements specifically described in the following.

The network interface 107 is for performing communication with anexternal device outside the playback apparatus 200, and is capable ofaccessing a server accessible via the Web, and accessing a serverconnected by a local network. For example, the network interface 107 isused for downloading BD-ROM additional contents publicized on the web.The BD-ROM additional content is content that is not stored in anoriginal BD-ROM, examples of which are additional sub audio, subtitles,special features and an application. It is possible to control thenetwork interface 107 from the BD-J platform, and to download, in thelocal storage 111, the additional content publicized on the web.

As stated in the above, the front end unit 101 includes the BD drive 110and the local storage 111, for example.

The BD drive 110 includes, for example, a semiconductor laser (notshown), collimator lenses (not shown), a beam splitter (not shown), anobjective lens (not shown), a condenser lens (not shown), an opticalhead (not shown) including a light detector (not shown). Light beamoutputted from the semiconductor laser is collected on a informationside of the optical disc through the collimator lenses, the beamsplitter and the objective lens. The collected light beam is reflectedand diffracted on the optical disc, and then collected by the lightdetector through the objective lenses, the beam splitter and thecondenser lenses. The generated signal corresponds to data read from theBD-ROM in accordance with the amount of light collected on the lightdetector.

The local storage 111 includes a built-in media and a removable media,and is used for storing the downloaded additional contents and data usedby the application. A storage area for the additional contents isprovided for each BD-ROM, and an area that can be used for storing datais provided for each application. Also, merge management information,which is a merge rule regarding how the downloaded additional contentsare merged with the data on the BD-ROM, is also stored in the built-inmedia and the removable media.

The built-in media is a writable recording medium such as a hard diskdrive and a memory built in the playback apparatus 200.

The removable medium is a portable recording medium, for example, and ispreferably a portable semiconductor memory card such as an SD card.

A description is given taking a case where the removable media is asemiconductor memory card as an example. The playback apparatus 200 isprovided with a slot (not shown) into which the removable media isinserted, and an interface (e.g. memory card I/F) for reading theremovable area Inserted into the slot. When the semiconductor memory isinserted into the slot, the removable media and the playback apparatus200 are electrically connected to each other, and it is possible toconvert data recorded in the semiconductor memory into an electricalsignal and read the electrical signal with use of the interface (e.g.memory card I/F).

The elements in the front end unit 101, the system LSI, the nonvolatilememory 105, the memory device 13, the back end unit and the hostmicrocomputer 106 are further described. FIG. 5 shows the structure ofthe front end unit 101, the system LSI, the nonvolatile memory 105, thememory device 13, the back end unit and the host microcomputer 106 indetail. As shown in FIG. 5, the front end unit 101, the system LSI, thenonvolatile memory 105, the memory device 13, the back end unit and thehost microcomputer 106 include a read buffers 1 and 2, a virtual filesystem 3, a demultiplexer 4, the video decoders 5 a and 5 b, a videoplane 6, image decoders 7 a and 7 b, image memories 7 c and 7 d, animage plane 8, an audio decoder 9, an interactive graphics plane 10, abackground plane 11, the playback state/setting register (PSR) set 12, astatic scenario memory 13, the playback control engine 14, thecomposition unit 15, an HDMI transmission/reception unit 16, aleft-right processing storage unit 17, a display function flag holdingunit 18, a plane shift engine 19, the offset setting unit 20, the BD-Jplatform 22, a dynamic scenario memory 23, a mode management module 24,an HDMV module 25, a UO detection module 26, a still image memory 27 a,a still image decoder 27 b, a display mode setting initial displaysetting unit 28 and a display mode storage unit 29.

The read buffer 1 temporarily stores source packets composing extentsthat compose the base view stream read from the BD drive 110. The readbuffer 1 transfers the source packets to the demultiplexer 4 afteradjusting the transfer speed, and has a size “RB1” as stated in theabove.

The read buffer 2 stores source packets composing extents that composethe enhanced view stream read from the BD drive 110. The read buffer 2transfers the source packets to the demultiplexer after adjusting thetransfer speed, and has a size “RB2” as stated in the above.

The virtual file system 3 configures a virtual BD-ROM (virtual package)in which the additional contents stored in the local storage are mergedwith the contents on the loaded BD-ROM based on the merge managementinformation downloaded in the local storage 111 together with theadditional contents.

The virtual package and the original BD-ROM can be referred to from acommand interpreter which is a main operational part in the HDMV mode,and the BD-J platform which is a main operational part in the BD-J mode.The playback apparatus 200 performs the playback control with use of thedata on the BD-ROM and the data on the local storage during the playbackof the virtual package.

The demultiplexer 4 is composed of a source packet depacketizer and aPID filter. Receiving an instruction from a packet identifiercorresponding to a stream to be played back (the stream is included inthe structured virtual package (data on the loaded BD-ROM and the localstorage corresponding to the loaded BD-ROM), the demultiplexer 4executes packet filtering based on the packet identifier. In executingthe packet filtering, the demultiplexer 4 extracts one of the left-viewvideo stream and the right-view video stream that corresponds to adisplay method flag based on the flag in the left-right processingstorage unit 31, and the demultiplexer 4 transfers the video stream tothe video decoder 5 a or the video decoder 5 b.

When a stream separated from the stream to be played back is a subtitlestream, the demultiplexer 4 writes the separated subtitle stream in tothe image memory. When the subtitle streams (a left-view subtitlestream, and a right-view subtitle stream) are included in the stream,the demultiplexer 4 writes the left-view subtitle stream in the imagememory 7 c, and writes the right-view subtitle stream in the imagememory 7 d.

When the 2D subtitle stream (subtitle stream used for the planardisplay) is included in the stream, the demultiplexer 4 writes the 2Dsubtitle stream in the image memory 7 c.

The video decoder 5 a decodes a TS packet outputted from thedemultiplexer 4, and writes a compressed picture in a left-eye plane 6(expressed as a code (L) in the video plane 6 in FIG. 5).

The video decoder 5 b decodes the enhanced view video stream outputtedfrom the demultiplexer 4, decodes the TS packet, and writes theuncompressed picture in a right view video plane 6 (expressed as a code(R) in the video plane 6 in FIG. 5).

The right-view video plane 6 is a plane memory that can store picturedata having a resolution such as 1920*2160 (1280*1440). The video plane6 has a left-eye plane (expressed as the code (L) in the video plane 6in FIG. 5) having an area Capable of storing data with resolution suchas 1920*1080 (1280*720), and a right-eye plane (expressed as the code(R) in the video plane 6 in FIG. 5) having an area capable of storingdata with resolution such as 1920*1080 (1280*720).

When the display mode of the video plane is in 3D display mode, and thestereo mode is ON, the video decoder 5 a decodes the left-view videostream, and writes the decoded left-view video stream in the left-eyeplane (expressed as the code (L) in the video plane 6 in FIG. 5). Thevideo decoder 5 b decodes the right-view video stream, and writes thedecoded right-view video stream in the right-eye plane (expressed as thecode (R) in the video plane 6 in FIG. 5).

When the display mode of the video plane is a 3D display mode, and thestereo mode is OFF, the video decoder 5 a decodes the left-view videostream, for example, and writes the decoded left-view video stream inthe left-eye plane (expressed as the code (L) in the video plane 6 inFIG. 5) and in the right-eye plane (expressed as the code (R) in thevideo plane 6 in FIG. 5).

When the display mode of the video plane is a 2D display mode, thedemultiplexer 4 transmits the 2D video stream to the video decoder 5 a,and the video decoder 5 a writes the decoded 2D video data held in theleft-eye plane (expressed as the code (L) in the video plane 6 in FIG.5).

Although an examples are shown for cases where each of the left-eyeplane and the right-eye plane included in the video plane 6 shown inFIG. 5 is a physically separated memory, the structure of the videoplane 6 is not limited to this. Therefore, areas for the left-eye planeand the right-eye plane may be provided together as one memory. In suchcase, the video data is written in each of the corresponding areas (leftand right).

Each of the image decoders 7 a and 7 b decodes TS packets composing thesubtitle stream that is outputted from the demultiplexer 4 and writtenin the image memories 7 c and 7 d, and writes the uncompressed graphicssubtitles in the graphics plane 8 a. The “subtitle streams” decoded fromthe image decoders 7 a and 7 b are data pieces each showing subtitlescompressed by a run-length coding, and is defined by pixel codes showinga Y value, a Cr value, a Cb value and an α value and a run lengths ofthe pixel codes.

The image plane 8 is a graphics plane capable of storing graphics data(e.g. subtitle data) obtained by for example, decoding the subtitlestream with a resolution of 1920*1080 (1280×720). The image plane 8 hasa left-eye plane (expressed as a code (L) in the image plane 8 in FIG.5) having an area capable of storing data having a resolution of1920*1080 (1280*720), for example, and a right-eye plane (expressed as acode (R) in the image plane 8 in FIG. 5) having an area capable ofstoring data having a resolution of 1920*1080 (1280*720), for example.

When the display mode of the plane with subtitle data is in 3D displaymode, and the stereo mode is ON, the image decoder 7 a decodes theleft-view subtitle stream stored in the image memory 7 c, and writes thedecoded subtitle stream in the left-eye plane (expressed as a code (L)in the image plane 8 in FIG. 5). The image decoder 7 b decodes theright-view subtitle stream stored in the image memory 7 d, and writesthe decoded right-view subtitle stream in the right-eye plane (expressedas a code (R) in the image plane 8 in FIG. 5).

When the display mode of the plane with subtitle data is a 3D displaymode, and the stereo mode is OFF, the image decoder 7 a decodes theleft-view subtitle stream stored in the image memory 7 c, and writes thedecoded left-view subtitle stream in the left-eye plane (expressed as acode (L) in the image plane 8 in FIG. 5) and in the right-eye plane(expressed as a code (R) in the image plane 8 in FIG. 5).

When the display mode of the plane with subtitle data is a 2D displaymode, the demultiplexer 4 stores the 2D subtitle stream in the imagememory 7 c, and the image decoder 7 a decodes the 2D subtitle streamstored in the image memory 7 c, and writes the left-eye video plane(expressed as a code (L) in the image plane 8 in FIG. 5).

Although an examples are shown for cases where each of the left-eyeplane and the right-eye plane included in the image plane 8 shown inFIG. 5 is a physically separated memory, the structure of the imageplane 8 is not limited to this. Therefore, areas for the left-eye planeand the right-eye plane may be provided together as one memory. In suchcase, the corresponding graphics data is written in each of thecorresponding areas (left and right).

The audio decoder 9 decodes audio frames outputted from thedemultiplexer 4, and outputs the uncompressed audio data.

The interactive graphics plane 10 is a graphics plane having a storagearea capable of storing graphics data written by the BD-J applicationusing the rendering engine 21 with resolutions such as 1920*2160(1280*1440). The interactive graphics plane 10 has, for an example, aleft-eye plane (expressed as a code (L) in the interactive graphicsplane 10 in FIG. 5) having an area capable of storing data having aresolution of 1920*1080 (1280*720), and a right-eye plane (expressed asa code (R) in the interactive graphics plane 10 in FIG. 5) having anarea capable of storing data having a resolution of 1920*1080(1280*720).

When the display mode of the interactive graphics plane is in 3D displaymode, and the stereo mode is ON, it is indicated that the BD-Japplication includes a program that writes an interactive graphics thatis viewable by the left eye (left-eye interactive graphics) and aninteractive graphics that is viewable by the right eye and is differentfrom the left-eye interactive graphics (right-eye interactive graphics).

The left-eye interactive graphics and the right-eye interactive graphicswritten by this rendering program can be seen from different angles soas to allow the viewer to see stereoscopic graphics.

When the left-eye interactive graphics and the right-eye interactivegraphics are displayed, the BD-J application writes the left-viewinteractive graphics in the left-eye plane (to which a code (L) is givenin the interactive graphics plane 10 in FIG. 5) with use of therendering engine 21, and writes the right-view interactive graphics inthe right-eye plane (to which a code (R) is given in the interactivegraphics plane 10 in FIG. 5).

When the display mode of the interactive graphics plane is in 3D displaymode, and the stereo mode is OFF, the BD-J application writes theleft-view interactive graphics in the each of the planes to which thecode (L) and the code (R) are respectively given, with use of therendering engine 21.

When the display mode of the interactive graphics plane is in 2D displaymode, the BD-J application writes the 2D interactive graphics in theinteractive graphics plane 10 (more specifically, the plane to which thecode (L) is given in the interactive graphics plane 10) with use of therendering engine 21.

Although examples are shown for cases where a left-eye area (to whichthe code (L) is given) and a right-eye area (to which the code (R) isgiven) of the interactive graphics plane 10 shown in FIG. 5 are providedin one plane memory, the structure of the interactive graphics plane 10plane is not limited to this. Therefore, the left-eye area (to which thecode (L) is given) and the right-eye area (to which the code (R) isgiven) of the interactive graphics plane 10 may be physically separatedfrom one another.

The “graphics data” held in the interactive graphics plane 10 isgraphics whose pixels each is defined by an R value, a G value, a Bvalue, and an α value. The graphics written in the interactive graphicsplane 10 is an image or a widget mainly used for composing the GUI.

Although the image data and the graphics data are different in terms ofstructure, they are collectively expressed as graphics data. There aretwo types of the graphics plane (i.e. the image plane 8 and interactivegraphics plane 10). Hereafter, when the term “graphics plane” is used,it referrers to both or one of the image plane 8 and the interactivegraphics plane 10.

The still image memory 27 a stores still image data that is extractedfrom the structured virtual package and is to be a background image.

The background plane 11 is a plane memory capable of storing the stillimage data to be a background image having a resolution such as1920*2160 (1280*1440). Specifically, the background plane 11 has aleft-eye plane (expressed as a code (L) in the background plane 11 inFIG. 5) having an area capable of storing data having a resolution of1920*1080 (1280*720), and a right-eye plane (expressed as a code (R) inthe background plane 11 in FIG. 5) having an area capable of storingdata having a resolution of 1920*1080 (1280*720).

When the display mode of the background plane is in 3D display mode, andthe stereo mode is ON, the still image decoder 27 a decodes left-viewstill image data and right-view still image data that are stored in thestill image memory 27 a. Then the image decoder 27 a writes theleft-view still image data and the right-view still image data held inthe left-eye plane (to which a code (L) is given in the background plane11 shown in FIG. 5) and the right-view plane (to which a code (R) isgiven in the background plane 11 shown in FIG. 5), respectively.

When the display mode of the background plane is in 3D display mode, andthe stereo mode is OFF, the background plane 11 decodes the left-viewstill image data from among the 3D background images stored in the stillimage memory 27 a (the left-view still image data and the right-viewstill image data), and writes the decoded left-view image data held inthe left-eye plane (to which a code (L) is given in the background plane11 shown in FIG. 5) and in the right-eye plane (to which a code (R) isgiven in the background plane 11 shown in FIG. 5).

When the display mode of the background image is a 2D display mode, thestill image decoder 27 b decodes the 2D still image data stored in thestill image decoder 27 a, and writes the decoded 2D still image dataheld in the left-eye plane (to which a code (L) is given in thebackground plane 11 shown in FIG. 5).

Although examples are shown for cases where the left-eye plane (to whichthe code (L) is given) and a right-eye plane (to which the code (R) isgiven) of the background plane 11 shown in FIG. 5 are provided in oneplane memory, the structure of the background plane 11 is not limited tothis. Therefore, the left-eye plane (to which the code (L) is given) andthe right-eye plane (to which the code (R) is given) of the backgroundplane 11 may be physically separated from one another.

Note that although it is disclosed that each of the video plane 6, theimage plane 8, the interactive graphics plane 10 and the backgroundplane 11 as shown in FIG. 5 is provided with a storage area for storingthe left-eye data and a storage area for storing the right-eye data, thestructures of these planes are not limited to this. Therefore, each ofthe planes may have only one memory area that is alternatively used as aleft-eye area and a right-eye area.

The PSR set 12 is a collection of registers including a playback stateregister storing therein information on playback states of playlists, aplayback setting register storing configuration information showing aconfiguration in the playback apparatus 200, and a general registercapable of storing arbitrary information used by the contents. Each ofthe playback states of the playlists shows which of the AV data piecesin each kind of AV data information pieces that are written in theplaylist are used, and at which position (time point) of the playlistthe playback is performed.

When the state of each of the playlists changes, the playback controlengine 14 stores information on what has been changed in the PSR set 12.Also, according to an instruction from the application executed by thecommand interpreter which is a main operational part in the HDMV mode orthe main operational part in the BD-J mode, the playback control engine14 is capable of storing a value specified by the application, andtransferring the stored value to the application.

The static scenario memory 13 is a memory for storing current PlayListinformation or a current clip information. The current PlayListinformation is a current processing target from among a plurality ofpieces of PlayList information accessible from the BD-ROM, a built-inmedia drive or a removable media drive. The current clip information isa current processing target from among a plurality of pieces of clipinformation accessible from the BD-ROM, a built-in media drive or aremovable media drive.

The playback control engine 14 executes an AV playback function and aplayback function of the playlist in response to a function call fromthe command interpreter which is the main operational part in the HDMVmode and the Java platform which is the main operational part in theBD-J function. The AV playback function is a set of functions used inDVD players and CD players, and includes playback start, playback stop,pause, release of pause, release of freeze frame function, fastforwarding at a playback speed specified by an immediate value, fastrewinding at a playback speed specified by an immediate value, audioconversion, sub image conversion, and angle conversion. The playlistplayback function is to perform playback start or playback stop fromamong the above-stated AV playback functions according to the currentPlayList information composing current playlists, and current clipinformation.

When a disc is inserted into the medium, a playlist and an AV streamthat are playback processing targets by the playback control engine 12are a auto start playlist (AutoStartPlaylist) and a default start streamrespectively that are written in the current scenario on the BD-ROM. Theplayback of the AV stream starts due to a user operation (e.g. playbackbutton) or automatically done by event triggered by the terminal (i.esuch as resident application).

The composition unit 15 composites data held in the interactive graphicsplane 10, data held in the image plane 8, data held in the video plane 6and data held in the background plane 11.

Each of the interactive graphics plane 10, the image plane 8, the videoplane 6 and the background plane 11 has a separate layer structure. Dataheld in each of the planes is composited (overlaid) in order of thebackground plane 11, the video plane 6, the image plane 8, then theinteractive graphics plane 10. That is, even in the case where theplanar graphics is composited with the stereoscopic video, thecomposition unit 15 composites video held in the video plane 6 with thebackground image held in the background plane 11, then composites thesubtitles held in the image plane 8, then at last composites thegraphics held in the interactive graphics plane 10. The composited imageis displayed as a result. If done otherwise, graphics part may be hiddenby the video, and thus will look unnatural.

The composition unit 15 also includes a scalar function. When thecomposition unit 15 composites shifted data pieces in the planes, thecomposition unit 15 is capable of performing scaling to make the imageappear to be smaller or larger.

The purpose of performing the scaling is described in the following. Inthe real world, close objects appear to be larger, and distant objectsappear to be smaller. However, just shifting the above-stated image datamakes the object appear to be close to or distant from the viewer withits size unchanged. In such case, the viewer possibly may feeluncomfortable. The scaling is performed for the purpose of reducing sucha sense of discomfort. One example, would be to in order to display theimage closer to the viewer with a large shift amount, the subtitles heldin the image plane 8 and the image held in the interactive graphicsplane 10 can be enlarged at the timing of scaling.

The HDMI transmission/reception unit 16 includes an interface thatcomplies with the HDMI standard (HDMI: High Definition MultimediaInterface). The HDMI transmission/reception unit 16 performstransmission and reception such that the playback apparatus 200 and adevice (in this example, a TV 400) that performs the HDMI connectionwith the playback apparatus 200 comply with the HDMI standard. Thepicture data stored in the video and audio data decoded from theuncompressed audio data by the audio decoder 9 are transmitted to the TV400 via the HDMI transmission/reception unit 16. The TV 400 holdsinformation such as, whether the TV 400 is capable of displaying datastereoscopically, information regarding-resolutions at which the planardisplay can be performed, and information regarding resolutions at whichthe stereoscopic display can be performed. When the playback apparatus200 gives a request via the HDMI transmission/reception unit 16, the TV400 gives the playback apparatus 200 necessary information (e.g.information regarding whether the TV 400 is capable of displaying datastereoscopically, information regarding resolutions at which the planardisplay can be performed, and information regarding resolutions at whichthe stereoscopic display can be performed) requested by the TV 400.Thus, the playback apparatus 200 is capable of obtaining, from the TV400, the information regarding whether the TV 400 is capable ofdisplaying data stereoscopically via the HDMI transmission/receptionunit 16.

The left-right processing storage unit 17 stores information showingwhether the current output processing is for left-view video orright-view video. A flag in the left-right processing storage unit 17shows whether or not data to be outputted to a display device (TV inFIG. 1) connected to the playback apparatus 200 shown in FIG. 1 is theleft-view video or the right-view video. While the left-view video isoutputted, the flag in the left-right processing storage unit 17 is setas the left-view output. Also, while the right-view video is outputted,the flag in the left-right processing storage unit 17 is set as theright-view output.

The display function flag holding unit 18 stores a 3D display functionflag showing whether the playback apparatus 200 is capable of performingthe 3D display or not.

The plane shift engine 19 shifts coordinates of each of pixel datapieces held in the image plane 8 and/or coordinates of each of the pixeldata pieces in the interactive graphics plane 10 in a predetermineddirection (e.g. in a horizontal direction on a display screen) based onthe flag in the left-right processing storage unit 31 and depthinformation on the data held in the graphics plane. That is, even if theobjects of the graphics such as subtitles and the GUI used for data heldin the image plane 8 and the data held in the interactive graphics plane10 are not materials for the stereoscopic viewing, it is possible toobtain an effect that the objects are displayed in a position closer tothe viewer than a position of the display screen. When the viewer wantsonly the graphics to have a stereoscopic effect with video displayed as2 dimension, the composition unit 15 uses the left-view video for boththe left view and the right view instead of using the set of theleft-view video and the right-view video, and then composites theshifted data held in the image plane 8 and the shifted data held in theinteractive graphics plane 10 with the video.

The shifting targets do not have to be both the data held in the imageplane 8 and the data held in the interactive graphics plane 10. Shiftingtarget can be done for only one of the planes, either the data held inthe interactive graphics plane 10 or the data held in the image plane 8.

The plane shift engine 19 includes a storage area for storing a “planeoffset” (offset value) showing a direction in which data is shifted,along with distance by which the data is shifted, for performing theshifting. For example, when the playback apparatus 200 includes a setupfunction that can set a plane offset value, the plane shift engine 19stores a value set using the setup function. The playback apparatus 200may have two offset values, one for the image plane 8, and the other forthe interactive graphics plane 10, and chooses one of the offset valuesfor use according to a shifting target. When the playback apparatus 200does not have the setup function, “0” may be specified as a default (inthis case, graphics such as the subtitles and the GUI are displayed inthe position of the display screen, and there is no effect that anobject pops out from the display screen).

The rendering engine 21 includes base software (e.g. Java 2D, OPEN-GL),and writes graphics and a character string in the interactive graphicsplane 10 in accordance with the instruction from the BD-J platform 22 inthe BD-J mode. Also, in the HDMV mode, the rendering engine 21 writesgraphics data (e.g. graphics data corresponding to an input button)extracted from the graphics stream other than a stream corresponding tothe subtitles (subtitle stream), and writes the extracted graphics dataheld in the interactive graphics plane 10.

The BD-J platform 22 is a Java platform which is a main operational partin the BD-J mode. The BD-J platform 22 is fully provided with the Java2Micro Edition (J2ME) Personal Basis Profile (PBP 1.0) and GloballyExecutable MHP specification (GEM1.0.2) for package media targets. TheBD-J platform 22 reads byte codes from a class file in the JAR archivefile, and stores the heap memory to start the BD-J application. Then,the BD-J platform 22 converts byte codes composing the BD-J applicationand byte codes composing a system application into native codes, andcauses the MPU to execute the native codes.

The dynamic scenario memory 23 stores currant dynamic scenario, and isused for processing by the HDMV module which is the main operationalpart in the HDMV mode, and the Java platform which is the mainoperational part in the BD-J mode. The current dynamic scenario is acurrent execution target which is one of Index.bdmv, the BD-J object andthe movie object recorded in the BD-ROM, the built-in media, or theremovable media.

The display mode management module 24 stores Index.bdmv read from theBD-ROM, the built-in media, or the removable media, and performs modemanagement and branching control. The mode management by the modemanagement module 24 is to perform allocation of the dynamic scenario tothe module (i.e. to cause one of the BD-J platform 22 and the HDMVmodule 25 to execute the dynamic scenario).

The HDMV module 25 is a DVD virtual player to be a main operational partin the HDMV mode, and is a main execution part. This module includes acommand interpreter, and executes control of the HDMV mode by readingand executing the navigation commands composing the movie object. Thenavigation commands are written by a syntax similar to a syntax for theDVD-Video. Therefore, the playback control like the DVD-Video can berealized by executing these navigation commands.

The UO detection module 26 receives the user operation on the GUI. Theuser operation received by the GUI includes title selection determiningwhich of the titles recorded in the BD-ROM is selected, subtitleselection and audio selection. In particular, one of the user operationsunique to the stereoscopic playback is to receive the depth ofstereoscopic video. For example, there are three levels of the depthsuch as distant, usual and close, or levels of the depth may beexpressed by the numerical values such as how many centimeter or howmany millimeter.

The still image memory 27 a stores still image data read from the BD-ROM(or configured virtual package).

The still image decoder 27 b decodes still image data read from the readbuffer 27 a, and writes the uncompressed background image data held inthe background plane 11.

The display mode storage unit 29 stores information on a display modeand information on a stereo mode. When the 3D display function flag ofthe playback apparatus 200 shows that the playback apparatus 200 iscapable of displaying 3D video, the display mode which is a terminalsetting stored in the display mode storage unit 29 may be switched toone of a 2D mode and a 3D mode. Hereinafter, a state of the display modeshown as “3D” is referred to as a “3D display mode”, and a state of thedisplay mode shown as “2D” is referred to as a “2D display mode”.

When the playback apparatus 200 is in the 3D playback mode, the stereomodes of the planes are either ON or OFF. The difference between ON andOFF of the stereo modes affects compositing methods for the planes.

“Stereo mode ON” is a 3D display mode in which composition is performedsuch that the playback apparatus 200 displays the left-view and theright-view that look different.

“Stereo mode OFF” is a 3D display mode in which composition is performedsuch that the playback apparatus 200 displays the left-view and theright-view that look the same. That is, when viewed by both of the eyes,the picture does not look stereoscopic (planar). However, when the dataheld in the graphics plane 8 is shifted in the horizontal direction bythe plane offset, the plane graphics data (subtitle data) that is heldin the graphics plane 8 is to be displayed can be displayed in aposition closer to the viewer than a position of the display screen, orin a position more distant from the viewer than a position of thedisplay screen. The same effect can be obtained when offsets of videodata held in the video plane 6, interactive graphics data held in theinteractive graphics plane 10, and background image data held in thebackground plane 11 are adjusted when the “stereo mode is in the OFFstate”.

As described in the above, there are two modes, “Stereo mode ON” and“Stereo mode OFF” in the “3D display mode”. When the playback apparatus200 is in the “Stereo mode ON” state in the 3D display mode, theleft-view data and the right-view data (e.g. an image viewed by the lefteye and the image viewed by the right eye can be seen from differentangles) are held in the left-eye plane and the right-view plane,respectively, and displayed in accordance with the SYNC signal. Thismakes it possible to display the stereoscopic image.

Also, when the playback apparatus 200 is in the “Stereo mode OFF” statein the 3D display mode, one of the left-view data and the right-viewdata (the left-view data held in the present embodiment) is held in eachof the left-eye plane and the right-view plane, and the offsets of thestored data pieces are adjusted. This makes it possible to display theplanar image in a position closer to or more distant from the viewerthan the position of the display screen.

In the present embodiment, the “Stereo mode ON” and the “stereo modeOFF” can be set for each plane (i.e. the video plane 6, the graphicsplane 8, the interactive graphics plane 10 and the background plane 11).

The “2D display mode” is a normal display that displays the image in aposition corresponding to the position of the display screen. In suchcase, a decoder and a plane used in a default setting is predetermined,and the composited image is displayed with use of the decoder and theplane.

For example, when the playback apparatus 200 is in the “2D displaymode”, the composition unit 15 composites: the 2D video data written bythe video decoder 5 a in the left-eye video plane (expressed as the code(L) in the video plane 6 in FIG. 5); the 2D graphics data (subtitledata) written by the image decoder 7 a in the left-eye plane (expressedas the code (L) in the image plane 8 in FIG. 5); the 2D interactivegraphics written by the BD-J application in the left-eye plane(expressed as the code (L) in the interactive graphics plane 10 in FIG.5) using the rendering engine 21; and the still image data written bythe still image decoder 27 b in the left-eye plane (expressed as thecode (L) in the background plane 11 in FIG. 5).

At this time, the composition unit 15 performs the composition in theorder of the 2D still image data, the 2D video data, the 2D graphicsdata (subtitle data) and the 2D interactive graphics data in order fromthe data from the bottom. The composition may be performed withoutcompositing the 2D still image data when the video data is displayed onthe whole screen. The flag in the display mode storage unit 29 showingwhether the playback apparatus 200 is in the 2D display mode or the 3Ddisplay mode may also be stored in the playback state register 12, ormay be stored in both the display mode storage unit 29 and the playbackstate register 12.

The display mode setting initial display setting unit 28, sets thedisplay mode and the resolutions based on the BD-J object in the currenttitle provided with the BD-J platform unit.

This concludes a description of the internal structure of the playbackapparatus 200. The following describes switching between the 2D displaymode and the 3D display mode in the present embodiment in detail.

FIG. 6 shows switching between the 2D display mode and the 3D displaymode. On the left side of FIG. 6, an output model in the 2D display modeis shown. In the plane structure on the left side of FIG. 6, only oneeach of the video plane 6, the image plane 8 (“Subtitle” in FIG. 6), theinteractive graphics plane 10, the background plane 11 and an output isprepared.

Therefore, the same data is used for the left view and the right view inthe 2D display mode. As a result, the same data is outputted.

On the right side of FIG. 6, an output model in the 3D display mode isshown. When the playback apparatus 200 is in the 3D display mode, thevideo plane 6, the image plane 8 (“Subtitle” in FIG. 6) and theinteractive graphics plane 10 are prepared for each of the left view andthe right view. The picture data and the graphics data to be played backare held in each of the video plane 6, the image plane 8 (“Subtitle” inFIG. 6), the interactive graphics plane 10 and the background plane 11for the left eye and the right eye.

Therefore, the left-view output and the right-view output are performedseparately in the 3D display mode. A different image can be provided forthe left eye and the right eye. As a result, it is possible to obtain a3D effect that the stereoscopic object in the screen appears to pop outcloser to the viewer due to the binocular disparity.

(Specification of the Stereoscopic Effect)

FIG. 7 shows one example of composition processing when stereo modes ofall the planes are ON, and when stereo modes of all the planes are OFF,in the 3D display mode.

Although FIG. 7 shows an example of a case where the stereo modes of theplanes are the same, ON/OFF of the stereo mode may be changed for eachof the planes.

On the left side of FIG. 7, the plane structure when the stereo modes ofall the planes are ON is shown. On the right side of FIG. 7, the planestructure when the stereo modes of all the planes are ON.

The first row shows the background plane 11 and the outputted databefore the composition.

The second row shows the video stream and the outputted data before thecomposition.

The third row shows the image plane 8 and the outputted data before thecomposition.

The fourth row shows the interactive graphics plane 10 and the outputteddata before the composition.

When the stereo mode is ON, a left-eye background plane which isexpressed as an area to which (L) is given is used for writing theleft-view background data, and a right-eye background plane which isexpressed as an area to which (R) is given is used for writing theright-view background data. Each of the background data pieces iscomposited with the corresponding left-eye or right-view picture. Whenthe stereo mode of the background plane 11 is OFF, the left-viewbackground data is written, by the application, in each of the areas towhich (L) and (R) are given respectively in the background plane 11.Therefore, the right-view background data does not affect the display.

When the stereo mode is ON, picture data of the left-eye video in thevideo stream is held in the left-view video plane. Also, picture data ofright-eye video in the video stream is held in the right-view videoplane. When the video plane 6 is in the OFF state of the stereo mode,the picture data of the left-eye video is held in both the left-viewvideo plane and the right-view video plane.

When the stereo mode is ON, in the image plane 8, left-view image datais written in a left-eye image plane expressed as the area to which (L)is given, and right-view image data is written in a right-eye imageplane expressed as the area to which (R) is given. Each of the imagedata pieces is composited with the corresponding left-eye or right-viewpicture.

When the image plane 8 is in the OFF state of the stereo mode, thesubtitle graphics corresponding to the right-view image data does notaffect the display. Also, when the stereo mode is OFF, the content inthe image plane 8 is a content shifted in the right or left direction(“Shifted Left” in FIG. 7).

When the stereo mode is ON, in the interactive graphics plane 10,left-view interactive graphics is written in a left-view interactivegraphics plane expressed as an area to which (L) is given, andright-view interactive graphics is written in a right-view interactivegraphics plane expressed as an area to which (R) is given. Each of theinteractive graphics data pieces is composited with the correspondingleft-eye or right-view picture.

When the interactive graphics plane 10 is in the OFF state of the stereomode, the right-view interactive graphics by the application does notaffect the display. Also, when the stereo mode is OFF, the content inthe interactive graphics plane 10 is a content that is shifted in theright or left direction (“Shifted Left” in FIG. 7).

FIG. 8 shows how the data held in the background plane 11, the data heldin the video plane 6, the data held in the image plane 8 and the dataheld in the interactive graphics plane 10 are composited with oneanother when the stereo modes of all the planes are ON. In the stereomode, it can be seen that left-view background data u4, left-view videou3 read from the video stream, left-view graphics u2 in the image plane8 and left-view graphics u1 in the interactive graphics plane 10 arecomposited as the left-view in this order.

Also, it can be seen that right-view background data u8, right-viewvideo u7 read from the video stream, right-view graphics u6 in the imageplane 8 and right-view graphics u5 in the interactive graphics plane 10are composited as the right view in this order.

FIG. 9 shows how the data held in the background plane 11, the data heldin the video plane 6, the data held in the image plane 8 and the dataheld in the interactive graphics plane 10 are composited with oneanother when the stereo modes of all the planes are OFF. When the stereomodes are OFF, it can be seen that left-view background data r4,left-view video r2 read from the video stream, Shifted Left graphics r3which is left-view graphics in the image plane 8 that has been shiftedin a predetermined direction (the right direction in FIG. 8) and ShiftedLeft graphics r1 which is left-view graphics in the interactive graphicsplane 10 that has been shifted in a predetermined direction (the rightdirection in FIG. 9) are composited as the right view in this order.

Also, it can be seen that right-view background data r8, left-view videor6 read from the video stream, Shifted Left graphics r7 which is theleft-view graphics in the image plane 8 that has been shifted in adirection opposite to the predetermined direction (the left direction inFIG. 8) and Shifted Left graphics r5 that is the left-view graphics inthe interactive graphics plane 10 that has been shifted in a directionopposite to the predetermined direction (the left direction in FIG. 8)are composited as the right view in this order.

In FIG. 7 to FIG. 9, offsets of the background data and the video werenot adjusted (i.e. an offset is 0, more specifically, the image isdisplayed in the position of the display screen), strictly for thepurpose of simplifying the above-stated description. Therefore, thesetting of the offsets is not limited to the above-stated descriptions,and the offsets may be adjusted such that the video is positioned in aposition more distant from the viewer than a position of the graphicsimage (subtitles), and the background data is in a position more distantfrom the viewer than a position of the background data.

The following describes switching between the stereo modes in thepresent embodiment.

FIG. 10 shows the composition result for each of the planes.

Each of 6L and 6R are examples of composition results for the videoplane 6. It can be seen from a difference in direction in which a womanfaces that images in a left-view stream and images a right-view streamare taken from different angles. Note that a difference in directions inwhich a man and the woman face and a difference in positions of the manand woman are schematic, and show neither accurate directions in whichthe man and woman face nor accurate positions of the man and woman forrealizing the stereoscopic display.

Subtitles “I love you” in the image plane 8 is an image obtained bydecoding the subtitle data by the image decoder.

A GUI part in a form of the sun in the interactive graphics plane 10 iswritten in the interactive graphics plane 10 by the BD-J application.

6LL is outputted left-view image after the composition, and 6RR isoutputted right-view image after the composition. It can be seen thatthe subtitles “I love you” is shifted in the right direction andcomposited in the left-view image of 6LL. Also, the subtitle “I loveyou” is shifted in the left direction in the right-view image of 6RR.

FIG. 11 shows one example of a case where an image output when thestereo modes of all of the planes are ON is displayed on the 3D display.

A right-view image and a left-view image are filtered through theshutter glasses 500, for example, appear differently from each other tocorresponding eyes. Observe that the subtitles “I love you” and the GUIpart in the faun of the sun are different in each of the right and leftimages in addition to the fact that the images of the video stream aremade stereoscopic by compositing the left-eye and the right-eye images.Thus, it can be seen that the depths of the video, the subtitles and theGUI are maintained naturally by turning the stereo mode ON when both theright-eye and left-eye contents are prepared in advance.

FIG. 12 shows an example of stereoscopic video that appears in a casewhere the viewer sees, through the shutter glasses 500, an imageoutputted when the stereo modes of the all the planes except for thevideo plane 6 are OFF. The stereo mode of the graphics plane needs to beOFF for discs that do not hold stereoscopic subtitles and GUIs (separateimages for both the left and right eye). The reason that a disc may notcontain stereoscopic subtitle/GUI may be that 1.) The disc does not havesufficient capacity to hold data for both eyes, or 2.) The content onthe disc was created for 2D viewing and stereoscopic data is notavailable. Since the stereo mode of the video plane 6 is ON in FIG. 12,it can be seen that images in the left-view stream and images in theright-view stream are taken from different angles. Meanwhile, the stereomodes of the image plane 8 and the interactive graphics plane 10 areOFF, and the subtitles and the GUI that are shifted in the right andleft directions are composited with the video in the video plane 6.Therefore, since the subtitles and GUI can be displayed closer to theviewer than the stereoscopic video even if only subtitles and GUI foronly one eye exists. This lessens the fatigue or stress caused to theeyes of the viewer.

(Control for Realizing the Stereoscopic Effect)

The following describes a direction in which the data held in thegraphics plane is shifted for realizing the stereoscopic effects.

Directions to which the plane shift engine 19 shifts the data held inthe graphics plane depends on whether the data held in the graphicsplane should stereoscopically appear in a position distant from theviewer than a position of the screen or should stereoscopically appearto pop out from the screen. In the present embodiment, it is presumedthat the left-view images are shifted in the right direction (i.e. thedata stereoscopically appears to pop out from the screen).

A difference between coordinates of each pixel data piece shifted in theright direction and coordinates of each pixel data piece shifted in theleft direction is called a “shift amount”. Thus, the shift amount iscalculated in a way that the outputted result controls the depth of theimage plane 8 or the data held in the interactive graphics plane 10.Here, the depth value shows how close to or distant from the viewer dataheld in each plane is displayed. The shift amount also can be calculatedwith use of one of parameters that can be adopted as the binoculardisparity between both of the eyes at a time of executing thestereoscopic playback.

The parameter for shifting the pixel data pieces in the graphics planein the right and left directions by the above-stated shift amount iscalled “plane offset”. While the shift amount is a scalar amount, theplane offset is a vector having a positive or negative value. The planeoffset indicates the direction, left direction or right direction, andthe amount of shifting to be shifted in pixels from the currentposition. In the following description, the shifting is executedaccording to the plane offset. The plane offsets can be represented asthe actual shift amount in pixels (with positive and negative value) orit can be represented as a value that can be used to calculate theactual shift amount in pixels.

The following describes the meanings of the positive and negative signsof the plane offset.

The plane offset of the graphics plane shows by how many pixels thecoordinates of each of the pixel data pieces held in the right-viewgraphics plane and the coordinates of each of the pixel data pieces heldin the left-view graphics plane are shifted.

When the viewer wants to obtain an effect that the video in the graphicsplane pops out from the screen, the plane shift engine 19 shifts thedata held in the left-view graphics plane in the right direction by theshift amount shown by the plane offset before the composition of datapieces in the planes. Then the plane shift engine 19 shifts the dataheld in the right-view graphics plane in the left direction by the shiftamount shown by the plane offset. In the present embodiment, an effectthat the images pops out from the screen can be obtained when the signof the plane offset is a “positive sign”.

When the viewer wants to obtain an effect that the images are displayedin a position distant from the viewer than the position of the screen,the plane shift engine 19 shifts the data held in the left-view graphicsplane in the left direction by the shift amount shown by the planeoffset before the composition of data pieces in the planes. Then theplane shift engine 19 shifts the data held in the right-view graphicsplane in the right direction by the shift amount shown by the planeoffset. In the present embodiment, an effect that the images aredisplayed in the position distant from the viewer than the position ofthe screen can be obtained when the sign of the plane offset is a“negative sign”.

When the plane offset is “0”, the plane shift engine 19 does not performthe shifting, which means that the plane shift engine 19 intentionallyomits the shifting.

FIG. 13A and FIG. 13B show Shifted Left graphics plane that are shiftedin the right direction, and Shifted Left graphics plane that are shiftedin the left direction, respectively.

As shown in FIG. 13A, as a result of shifting the image plane 8 to theright direction, a transparent area at the left end exists, and an endportion of the image plane is cropped. Similarly, there is a transparentarea at the left end of the interactive graphics plane 10 that isshifted in the right direction, and an end portion of the interactivegraphics plane is cropped.

As shown in FIG. 13B, as a result of shifting the image plane 8 to theleft direction, a transparent area at the right end exists, and the leftend portion of the image plane is cropped. Similarly, there is atransparent area at the right end portion of the interactive graphicsplane 10 that is shifted in the left direction, and the left end portionof the interactive graphics plane is cropped.

Each of FIG. 14A and FIG. 14B shows the internal structure of the imageplane 8. When the resolution is set to 1920*1080, the image plane 8 iscomposed of 8 bit-length memory elements, and has a size of 1920(horizontal)*1080 (vertical) as shown in FIG. 14A.

This means that the image plane 8 has a resolution of 1920*1080, and hasa memory allocation capable of storing an 8-bit pixel code for eachpixel. The 8′-bit pixel codes stored in the memory elements areconverted into Y, CR and Cb values by color conversion using the colorlook-up table. The correspondence relationship among the pixel codes andthe Y, CR and Cb values is defined by a pallet definition segment in thesubtitle data.

FIG. 14B shows pixel data pieces held in the image plane 8. As shown inFIG. 14B, the graphics data held in the image plane 8 is composed ofpixel data pieces that compose a foreground area (part composing thesubtitles “I love you”) and pixel data pieces that compose a backgroundarea. Pixel codes showing transparent colors are stored in the memoryelements corresponding to the background area. In this area, the videoin the video plane 6 can be seen through when the graphics data held inthe image plane 8 is composited with the video in the video plane 6.Meanwhile, pixel codes showing colors except for the transparent color(non-transparent color) are stored in the memory elements correspondingto the foreground area. The subtitles are written by the y, Cr, Cb and αvalues each showing the non-transparent color.

In the area corresponding to transparent pixels, the content in thebackground graphics plane 10 and the content in the video plane 6 can beseen through when the composition unit 15 composites the graphics dataheld in the image plane 8 with the data held in the background graphicsplane 11 and the video in the video plane 6. The transparent areacontributes to realizing the composition.

FIG. 15A, FIG. 15B and FIG. 15C show the pixel data pieces composing theforeground area and the pixel data pieces composing the background areaafter the shifting in the right direction or the shifting in the leftdirection have been performed. FIG. 15A shows the pixel data piecesbefore the shifting, and FIG. 15B shows the pixel data pieces after theshifting in the right direction is performed. It can be seen that a word“y” in the subtitles “I love you” disappears from the screen after theshifting when the shift amount is 15 pixels. FIG. 15C shows pixel datapieces after the shifting in the left direction is performed. It can beseen that “o” in the subtitles “you” succeeding the subtitles “I love”appears when the shift amount is 15 pixels.

This concludes the description of the internal structure of the imageplane 8, and the description of the arrangements of pixel data piecesbefore and after the plane shift engine 19 performs the shifting in theimage plane 8. The following describes the internal structure of theinteractive graphics plane 10, and the arrangements of pixel data piecesheld in the internal structure of the interactive graphics plane 10before and after the shifting.

Each of FIG. 16A and FIG. 16B shows the internal structure of theinteractive graphics plane 10. When the resolution is set to 1920*1080,the interactive graphics plane 10 is composed of 32 bit-length memoryelements, and has a size of 1920*1080 as shown in FIG. 16A. This meansthat the interactive graphics plane 10 has a resolution of 1920*1080,and has a memory allocation capable of storing a 32-bit R, G, B and αvalues per pixel. The 32-bit R, G, B and α values are composed of an8-bit R value, an 8-bit G value, an 8-bit B value and an 8-bittransparency α, respectively.

FIG. 16B shows pixel data held in the interactive graphics plane 10. Asshown in FIG. 16B, the graphics held in the interactive graphics plane10 is composed of pixel data corresponding to the foreground area shownby triangles expressing a corona of the sun, and other pixels aretransparent, meaning where corona of sun is not shown, the backgroundarea will be shown. Memory elements corresponding to the background areastore R, G, B and α values showing transparent colors. In this area, thevideo in the video plane 6 can be seen through when the graphics dataheld in the interactive graphics plane 10 is composited with the dataheld in the video plane 6. Meanwhile, memory elements corresponding tothe foreground area store R, G, B and α values showing thenon-transparent colors. The graphics are written with use of these R, G,B and α values showing the non-transparent colors.

The graphics data held in the background graphics plane 10, the video inthe video plane 6 and the graphics data held in the image plane 8 can beseen through when the foreground plane has a transparent pixel and thecomposition unit 15 composites these data pieces in the respectiveplanes. The transparent area contributes to realizing the composition.

This concludes the description of the pixel data pieces composing theforeground area and the pixel data pieces composing the background areaafter the shifting in the right direction and the shifting in the leftdirection have been performed.

FIG. 17A, FIG. 17B and FIG. 17C show the pixel data composing theforeground area and the pixel data composing the background area afterthe shifting in the right direction and the shifting in the leftdirection have been performed. FIG. 17A shows pixel data before theshifting is performed, and FIG. 17B shows pixel data after the shiftingin the right direction has been performed. In this case, it can be seenthat triangles showing a corona of the sun are shifted in the rightdirection. FIG. 17C shows pixel data after the shifting in the leftdirection is performed. It can be seen that the triangles showing thecorona of the sun are shifted in the left direction.

FIG. 18A FIG. 18B and FIG. 18C show the procedure for plane shiftingprocess of image plane 8.

FIG. 18A shows image plane that have been shifted in the left directionand image plane that have been shifted in the right direction.

FIG. 18B shows the shifting process of image plane to the rightdirection. As shown in FIG. 18B, horizontal shifting of image plane tothe right direction takes the following steps (1-1), (1-2) and (1-3).(1-1) The plane shift engine 19 crops a right end area of the imageplane 8. (1-2) The plane shift engine 19 shifts coordinates of each ofthe pixel data in the image plane 8 horizontally to the right directionby a shift amount shown by the plane offset. (1-3) The plane shiftengine 19 adds a transparent area at the left end of the image plane 8.

FIG. 18C shows the shifting process of image plane to the leftdirection. As shown in FIG. 18C, horizontal shifting of image plane tothe left direction takes the following steps (2-1), (2-2) and (2-3).(2-1) The plane shift engine 19 crops a left end area of the image plane8. (2-2) The plane shift engine 19 shifts the coordinates of the pixeldata in the image plane 8 horizontally to the left direction by a shiftamount shown by the plane offset. (2-3) The plane shift engine 19 adds atransparent area at the right end of the graphics data held in the imageplane 8.

Next, meanings of the positive and negative signs of plane offsets aredescribed.

When the playback apparatus 200 is in the 3D display mode and the stereomode of the image plane 8 is OFF, the playback apparatus 200 perform thecomposition after performing the following processing in the planesbased on the shift amount shown by the plane offsets.

When the positive sign of the plane offset is set, the plane shiftengine 19 shifts coordinates of each of the pixel data in the left-viewimage plane 8 in the right direction by the shift amount indicated bythe plane offset. Then, the plane shift engine 19 shifts coordinates ofeach of the pixel data in the right-view image plane 8 in the leftdirection by the shift amount indicated by the plane offset.

When the negative sign of the plane offset is set, the plane shiftengine 19 shifts coordinates of each of the pixel data in the left-viewimage plane 8 in the left direction by the shift amount indicated by theplane offset before the composition of the data in the respectiveplanes. Then, the plane shift engine 19 shifts coordinates of each ofthe pixel data pieces in the right-view image plane 8 in the rightdirection by the shift amount indicated by the plane offset.

FIG. 19A, FIG. 19B and FIG. 19C show the procedure for shifting theinteractive graphics plane 10.

FIG. 19A shows the interactive graphics plane that have been shifted inthe left direction and the interactive graphics plane that have beenshifted in the right direction.

FIG. 19B shows the shifting process of interactive graphics plane to theright direction. As shown in FIG. 19B, horizontal shifting ofinteractive graphics plane to the right direction takes the followingsteps (1-1), (1-2) and (1-3). (1-1) The plane shift engine 19 crops aright end area of the interactive graphics plane 10. (1-2) The planeshift engine 19 shifts the coordinates of the pixel data in theinteractive graphics plane 10 horizontally to the right direction by ashift amount indicated by the plane offset. (1-3) The plane shift engine19 adds a transparent area at the right end of the interactive graphicsplane 10.

FIG. 19C shows the shifting process of interactive graphics plane to theleft direction. As shown in FIG. 19C, horizontal shifting of interactivegraphics plane to the left direction takes the following steps (2-1),(2-2) and (2-3). (2-1) The plane shift engine 19 crops a left end areaof the interactive graphics plane 10.

(2-2) The plane shift engine 19 shifts the coordinates of each of thepixel data pieces in the interactive graphics plane 10 horizontally inthe left direction by a shift amount indicated by the plane offset.(2-3) The plane shift engine 19 adds a transparent area at the left endof the interactive graphics plane 10.

The following describes the size of the right end area and left end areaof the graphics data to be cropped, and the size of the areas to beadded at the right end and left end of the graphics data, in theinteractive graphics plane 10, at the time of shifting the coordinatesof each of the pixel data pieces in the respective planes. The planeoffset used for the above-stated shifting is a value that responds tothe binocular disparity between the right eye and the left eye. Thismeans that each of: the number of horizontal pixels composing the rightend area to be cropped; the number of horizontal pixels composing theleft end area of the graphics data to be cropped in the graphics plane;and the number of horizontal pixels composing the transparent area to beadded at each of the right and left ends of the graphics data held inthe graphics plane need to be the number of pixels corresponding to theplane offset.

Therefore, the number of horizontal pixels composing the right end areato be cropped and the number of horizontal pixels composing the left endarea to be cropped in the graphics plane is the number of pixelscorresponding to the shift amount indicated by the plane offset. Also,the number of vertical pixels composing the transparent areas is thenumber of pixels corresponding to the height of the graphics plane.

Similarly, the number of horizontal pixels composing the transparentareas to be added at each of the right and end ends of the graphicsimage held in the graphics plane is the number of pixels correspondingto the shift amount indicated by the plane offset. The number ofvertical pixels composing the transparent areas is the number of pixelscorresponding to the height of the graphics plane.

This concludes the description of the process of shifting the imageplane 8 and the interactive graphics plane 10. The following describesthe meanings of the positive and negative signs of the plane offset.

When the playback apparatus 200 is in the 3D playback mode, and thestereo mode of the interactive graphics plane 10 is OFF, the playbackapparatus 200 performs the composition after performing the followingprocess in the planes based on the plane offset.

When the positive sign of the plane offset is set, the plane shiftengine 19 shifts the left-view graphics plane in the right direction bythe shift amount shown by the plane offset. And, the plane shift engine19 shifts the right-view graphics plane by the shift amount shown by theplane offset.

When the negative sign of the plane offset is set, the plane shiftengine 19 shifts the left-view graphics plane in the left direction bythe shift amount shown by the plane offset. And, the plane shift engine19 shifts the right-view graphics plane in the right direction by theshift amount shown by the plane offset.

FIG. 20A and FIG. 20B show how the pop-out level of the subtitles (levelof how far the subtitles pop out from the display screen) changesaccording to the value of the plane offset.

An image depicting the graphics data that is closer to the viewer ineach of FIG. 20A and FIG. 20B is a graphics image for the right view(right-view graphics image) after shifting. An image depicting thegraphics data that is distant from the viewer in each of FIG. 20A andFIG. 20B is a graphics image for the left view (left-view graphicsimage) after shifting.

FIG. 20A shows a case where the sign of the plane offset is positive(the plane shift engine 19 shifts left-view graphics image in the rightdirection, and shifts the right-view graphics image in the leftdirection). When the sign of the plane offset is positive, subtitles tobe composited with the left-view video appears to be in a position moreto the right than the subtitles to be composited with the right-viewvideo. That is, since a point of convergence (focus position) is closerto the viewer than the position of the screen, the subtitles also appearto be closer to the viewer than the position of the screen.

FIG. 20B shows a case where the sign of the plane offset is negative.When the sign of the plane offset id negative, subtitles to becomposited with the right-view video appears to be in a position more tothe left than the subtitles to be composited with the right-view video.That is, since a point of convergence (focus position) is more distantfrom the viewer than the position of the screen, the subtitles alsoappear to be distant from the viewer than the position of the screen.

This concludes the description of the method for displaying thesubtitles in a position closer to the viewer than the position of thedisplay screen or in a position more distant from the viewer than theposition of the display screen by switching between the positive offsetand the negative offset.

FIG. 31A, FIG. 31B and FIG. 31C describe the principles of how the imageappear to be closer to the viewer than the display screen when the signof the plane offset is positive (the plane shift engine 19 shifts theleft-view graphics image in the right direction, and shifts theright-view graphics image in the left direction).

In each of FIG. 31A, FIG. 31B and FIG. 31C, a circle is an imagedisplayed on the display screen.

Firstly, since the image seen by the right eye and the image seen by theleft eye are in the same position when there is no offset, the focusposition at which both of the eyes see the image is on the displayposition (FIG. 31A).

Meanwhile, when the playback apparatus 200 is in the 3D display mode,and the stereo mode is OFF, the image seen by the left eye should appearin a position more to the right compared to the case where the offset is“0”. At this time, the right eye should be covered by the crystal liquidshutter glasses so that the right eye cannot see anything. The imageseen by the right eye, on the other hand, should be in a position moreto the left compared to the case where the offset is “0”. At this time,the left eye should be covered by the crystal liquid shutter glasses sothat the left eye cannot see anything (FIG. 31B).

We focus on the image with use of both of the eyes to recognize theimage in the focus position. Accordingly, when switching between a statein which the left eye sees the image and a state in which the right eyesees the image is performed by the crystal liquid shutter glasses at ashort time interval, both of our eyes try to focus on the image in aposition closer to us than the position of the display screen. As aresult, we have an illusion as if the image is in the focus positioncloser to us than the position of the display screen (FIG. 31C).

FIG. 42A, FIG. 42B and FIG. 42C describe the principles of how the imageappears to be distant from the viewer than the display screen when thesign of the plane offset is negative (the plane shift engine 19 shiftsthe left-view graphics image in the left direction, and shifts theright-view graphics image in the right direction).

In each of FIG. 42A, FIG. 42B and FIG. 42C, a circle is an imagedisplayed on the display screen. Firstly, since the image seen by theright eye and the image seen by the left eye are in the same positionwhen there is no offset, the focus position at which both of the eyessee the image is the position of display screen (FIG. 42A).

Meanwhile, when the playback apparatus 200 is in the 3D display mode,and the stereo mode is OFF, the image seen by the left eye should appearin a position more to the left compared to the case where the offset is“0”. At this time, the right eye should be covered by the crystal liquidshutter glasses so that the right eye cannot see anything. The imageseen by the right eye, on the other hand, should be in a position moreto the right compared to the case where the offset is “0”. At this time,the left eye should be covered by the crystal liquid shutter glasses sothat the left eye cannot see anything (FIG. 42B).

When switching between a state in which the left eye sees the image anda state in which the right eye sees the image is performed by thecrystal liquid shutter glasses at a short time interval, both of oureyes try to focus on the image in a position more distant from us thanthe position of the display screen. As a result, we have an illusion asif the image is in the focus position more distant from us than theposition of the display screen (FIG. 42C).

This concludes the description of the graphics image written in thegraphics plane. However, it is needless to say that the same concept ofthe above-stated offset may be applied to the interactive graphics plane10, the video plane 6 and the background plane 11.

(Shifting of the Coordinates of the Pixel Data in the respective MemoryElements in the Graphics Plane)

The following describes how the plane shift engine 19 shifts coordinatesof the pixel data in the memory elements in the graphics plane due tothe above-stated shifting. The graphics data is composed of pixel datapieces, and has a resolution 1920*1080 or 1280*720, for example.

FIG. 21 shows the pixel data pieces held in the graphics plane. In FIG.21, squared frames are memory elements having 32 or 8 bit word length.Hexadecimal numerical values such as 0001, 0002, 0003, 0004, 07A5, 07A6,07A7, 07A8, 07A9, 07AA and 07AB are addresses serially allocated tothese memory elements in the memory space of the MPU. Also, each of thenumerical values such as (0, 0), (1, 0), (2, 0), (3, 0), (1916, 0),(1917, 0), (1918, 0) and (1919, 0) in the respective memory elementsshows coordinates of which pixel data is stored in a correspondingmemory element.

The pixel data in a pair of coordinates (0, 0) is stored in the memoryelement whose address is 0001. The pixel data whose coordinates are (1,0) is stored in a memory element whose address is 0002. The pixel datawhose coordinates are (1918, 0) is stored in a memory element whoseaddress is 07A7. The pixel data whose coordinates are (0, 1) is storedin a memory element whose address is 07A9. That is, it can be seen thatgraphics data is stored such that a plurality of lines composing thegraphics are composed of memory elements having serially-arrangedaddresses. Thus, these pixel data pieces may be read in a burst mode bysequentially performing the DMA transfer on the memory elements to whichthe serial addresses are given.

Each of FIG. 22A and FIG. 22B shows what is held in the graphics planeafter the shifting has been performed.

FIG. 22A shows the graphics plane after the coordinates of each of thepixel data pieces have been shifted in the right direction with theplane offset being 3. Since the plane offset is “3”, the pixel datawhose coordinates are (0, 0) in a graphics plane coordinate system isstored in a memory element whose address is 0004, pixel data whosecoordinates are (1, 0) in the graphics plane coordinate system is storedin a memory element whose address is 0005, and pixel data whosecoordinates are (2, 0) in the graphics plane coordinate system is storedin a storage element whose address is 0006.

Also, it can be seen that pixel data whose coordinates are (0, 1) in thegraphics plane coordinate system is stored in a memory element whoseaddress is 07AC, pixel data whose coordinates are (1, 1) in the graphicsplane coordinate system is stored in a memory element whose address is07AD, and pixel data whose coordinates are (2, 1) in the graphics planecoordinate system is stored in a memory element whose address is 07AE.

FIG. 22B shows the graphics plane after the plane shift engine 19 hasshifted the coordinates of the pixel data pieces in the left directionwith the plane offset being 3. Since the plane offset is “3”, it can beseen that pixel data whose coordinates are (3, 0) in the graphics planecoordinate system is stored in a memory element whose address is 0001,pixel data whose coordinates are (4, 0) in the graphics plane coordinatesystem is stored in a memory element whose address is 0002, and pixeldata whose coordinates are (5, 0) in the graphics plane coordinatesystem is stored in a memory element whose address is 0003.

Also, it can be seen that pixel data whose coordinates are (3, 1) in thegraphics plane coordinate system is stored in a memory element whoseaddress is 07A9, pixel data whose coordinates are (4, 1) in the graphicsplane coordinate system is stored in a memory element whose address is07AA, and pixel data whose coordinates are (5, 1) in the graphics planecoordinate system is stored in a memory element whose address is 07AB.

As described in the above, it can be seen that, in the graphics planeafter the shifting has been performed, the coordinates of each of thepixel data in the graphics plane are shifted to the right or left fromthe original coordinates of each of the pixel data pieces by the numberof pixels shown by the plane offset.

In order to realize the shifting in the graphics plane, the plane shiftengine 19 needs to shift the coordinates of each of the pixel datacomposing the graphics data by an amount indicated in a predeterminedaddress, an address of a memory element on which each of the pixel datapieces composing the graphics data is positioned. It is needless to saythat the plane shift engine 19 can realize the shifting in the graphicsplane without changing the address of the memory element on which eachof the pixel data pieces is arranged, by performing processing thatcorresponds to the above-stated processing.

This concludes the description of what is held in the graphics planeafter the plane shift engine 19 has performed the shifting.

FIG. 23 shows an internal structure of the BD-J platform unit. As shownin FIG. 23, the BD-J platform is composed of a heap memory 31, a bytecode interpreter 32, a middleware 33, a class loader 34 and anapplication manager 35.

The heap memory 31 is a stack area on which the byte code of the systemapplication, the byte code of the BD-J application, a system parameterused by the system application and an application parameter used by theBD-J application are arranged.

The byte code interpreter 32 converts the byte codes composing the BD-Japplication stored in the heap memory 31 and byte codes composing thesystem application into native codes, and causes the MPU to execute theconverted codes.

The middleware 33 is an operating system for the embedded software, andis composed of a kernel and a device driver. The kernel provides theBD-J application with functions specific to the playback apparatus 200in accordance with calls of the application programming interface (API)from the BD-J application. Also, the kernel realizes hardware controlsuch as starting an interrupt handler unit by an interrupt signal.

The class loader 34 is one of the system applications, and loads theBD-J application by reading the byte codes from the class files thatexist in the JAR archive file, and storing the byte codes in the heapmemory 31.

The application manager 35 is one of the system applications, andperforms the application signaling of the BD-J application such asstarting the BD-J application or ending the BD-J application, based onthe application management table in the BD-J object.

This concludes the description of the internal structure of the BD-Jplatform unit.

In the above-stated layer model, the display mode setting initialdisplay setting unit 28 is in the lower layer of the platform unit,which sets a display mode and a resolution based on the BD-J object inthe current title used by the BD-J platform unit.

(Internal Structure of the Display Mode Storage Unit 29)

The display mode storage unit 29 can be referred to from theabove-stated layer model. Therefore, the display mode storage unit 29can be referred to from the layer model via the API, and includesinformation showing a setting and a state of each of the backgroundgraphics plane 11, the video plane 6, the image plane 8 and theinteractive graphics plane 10. The following describes the structure ofthe display mode storage unit 29, referring to FIG. 24.

FIG. 24 shows what is stored in the display mode storage unit 29.

In FIG. 24, the display mode storage unit 29 stores info nation on thesetting of the background plane 11, information on the setting of thevideo plane 6, information on the setting of the image plane 8,information on the setting of the interactive graphics plane 10, andinformation on the state of the display mode showing whether theplayback apparatus 200 is in the 2D mode or the 3D mode. “Resolution(:YY×ZZ in FIG. 24)” and a stereo mode (:ON or OFF in FIG. 24) and asetting of THREE_D (:ON or OFF in FIG. 24) are stored as setting itemsof each plane. Besides the above-stated setting items, the plane offsetmay be set in a range of “−63” to “+63” in the setting of the imageplane 8 and the setting of the interactive graphics plane 10.

The following describes the resolution supported by the display modestorage unit 29.

When the playback apparatus 200 is in the 2D display mode, the displaymode storage unit 29 supports a resolution of 1920*1080 pixels, aresolution of 1280*720 pixels, a resolution of 720*576 pixels, and aresolution of a 720*480 pixels for the background plane 11, the videoplane 6, the image plane 8 and the interactive graphics plane 10, as theinitial display setting. When the playback apparatus 200 is in the 3Ddisplay mode, the display mode storage unit 29 may support a resolutionof 1920*2160 pixels or a resolution of 1280*1440 pixels for each of thebackground plane 11, the image plane 8 and the interactive graphicsplane 10 in addition to the resolution of the 2D display mode. In suchcase, an aspect ratio of 1920*2160 pixels or an aspect ratio of1280*1440 pixels will be an aspect ratio of (16/18), and an upper halfof each plane is used as an area for the left eye, and a lower half ofeach plane is used as an area for the right eye.

This concludes the description of what is stored in the display modestorage unit 29.

The following describes the case where the THREE_D setting of each planechanges.

Firstly, it is necessary to change a setting of each plane in thedisplay mode storage unit 29 as condition when the display mode isswitched. For example, when the display mode is set to the 2D displaymode, the THREE_D setting of each plane needs to be OFF. Also, when thedisplay mode is set to the 3D display mode, the THREE_D setting of eachplane needs to be ON.

The following describes a case where a stereo mode and a plane offsetare reset.

When the playback apparatus 200 falls in the 2D display mode as a resultof the title selection, the stereo mode of each of the background plane11, the video plane 6, the image plane 8 and the interactive graphicsplane 10 is switched to OFF, and the shift amount of each of the imageplane 8 and the interactive graphics plane 10 shown by the offset is setto “0”.

When the display mode cannot be determined before the title selection,the display mode before starting the title is the 2D display mode.

The following describes what causes the switching between the displaymodes to be performed.

The display mode in the display mode storage unit 29 is possiblyswitched in one of two: a case where the playlist is switched, and acase where the BD-J application expressly gives an instruction for theswitching by a method call.

The following describes patterns of switching between the display modescaused by playback of the playlist.

The following describes the pattern in which the display mode is changedby the playback of the playlist. When the 2D playlist is played back inthe 3D display mode, the playback apparatus 200 automatically switchesthe display mode in the display mode storage unit 29 to the 2D displaymode. When the display mode is switched, an event is generatedasynchronously to the event listener registered in the BD-J application.

The display mode is not switched even if the playback of the playlist isstopped. That is, the display mode always remains in the current stateeven after the playback of the playlist is stopped.

The following describes the initial display setting by the display modesetting initial display setting unit 28.

The initial display setting is a value set “at the time of starting thetitle” of the display setting such as a resolution stored in the displaymode storage unit 29.

The following describes what affects the initial display setting.

When the 3D display function flag shows that the playback apparatus 200is capable of displaying 3D graphics, the initial display setting is notdetermined based on only the auto start playlist (playlist) of the 2Dplayback apparatus or the default setting described in the BD-J object.The determination of the initial display setting is also affected by the3D display mode before the title selection.

The following describes criteria for determining the initial displaysetting. The initial display setting is determined based on the defaultstart playlist.

When there is a auto start playlist, the playback apparatus 200 sets aresolution for the 2D graphics corresponding to a resolution of thevideo of the auto start playlist after automatically switching thedisplay mode to the 2D display mode when the playback apparatus 200 isin the 3D display mode, and the playlist is a 2D playlist.

When there is the auto start playlist, and the auto start playlist doesnot affect the display mode, the playback apparatus 200 selects theinitial display setting of a resolution corresponding to a resolution ofa video from among resolutions supported by the display mode before thetitle selection, and sets the resolution.

The following describes a case where the resolution and the plane offsetdo not change.

When there is the auto start playlist, and the playback apparatus 200switches to 3D display mode as a result of the title selection, stereomode values (described in the following) of the background plane 11, thevideo plane 6, the image plane 8 and the interactive graphics plane 10do not change, and the plane offset for each of these planes does notchange, either.

The following describes the criteria for determining the setting whenthere is no auto start playlist.

When there is no auto start playlist, the initial display setting isalways determined based on a value of the default resolution that iswritten in the BD-J object and corresponds to the selected title. Inthis case, the determination of the initial display setting is notaffected by the display mode before starting the title.

The following describes a case where the stereo mode and the planeoffset are reset.

When the title is selected in which the auto start playlist does notexist, the stereo mode of each of the image plane 8 and the interactivegraphics plane 10 is ON if the display mode is the 3D display mode, andthe stereo mode of each of the background plane 11 and the video plane 6is OFF.

Furthermore, when the display mode is switched from the 3D display modeto the 2D display mode, a stereo mode of each plane is set to OFF, andthe shift amount shown by the plane offset is set to “0”.

(Switching Between Display Modes when a Title is Changed)

As described in the above, there are information on resolutions andspecification of the auto start playlist for each of the BD-J objectscorresponding to the titles. Therefore, when a title is changed, thecorresponding BD-J object is also changed. Thus, the content of thedisplay mode set in the display mode storage unit 29 also changesaccordingly. The following describes how the display mode changes whenthe title is changed, with use of FIG. 25.

FIG. 25 shows, in a table, changes in the display modes when the titleis changed. The table in FIG. 25 shows the correspondence among thedisplay mode before selecting the current title, type of auto startplaylist for the title to be selected, default resolution written in theBD-J object, the display modes of the current title, and the initialdisplay settings.

The default playlists and the default resolutions are in the exclusiverelationship. If the auto start playlist exits in the title (i.e. thetitle is valid), the display mode responds to the resolution stipulatedin the auto start playlist. If the auto start playlist does not exist inthe title (i.e. the title is invalid), the display mode responds to theresolution stipulated in the auto start playlist.

The upper part CR1 of the table shows the display mode setting when theauto start playlist exists in the title (i.e. the title is valid), andthe lower part CR2 of the table shows the display mode setting when theauto start playlist does not exist in the title (i.e. the title isinvalid).

According to the contents written in the upper part of the table, whenthe immediately preceding display mode is a 3D display mode, and theauto start playlist is a 3D playlist, the display mode is set as the 3Doutput. In other cases, the display mode is set as the 2D output.

According to the contents written in the lower part of the table, whenthe immediately preceding display mode is a 2D display mode or a 3Ddisplay mode, and the default resolution is HD3D_(—)1920*1080, orHD3D_(—)1280*720, the display mode is set as the 3D output.

(Change in Display Modes in a Title)

A title is not always composed of one playlist such as auto startplaylist. The title is composed of a plurality of playlists in somecases. When the title is composed of a plurality of playlists, thesetting of the display mode in the display mode storage unit 29 isswitched when the playback of the playlist is changed. The followingdescribes how the display mode changes when the playlist changes in atitle.

FIG. 26 shows the switching between the display modes when the playlistbeing played back is changed in the title. The table in FIG. 26 showsthe correspondence among the current status, the action to be taken andthe final display mode. In this table, rows with the hatched lines eachshow a case where the final display mode is determined to be a 3Ddisplay mode. Each of the other rows shows a case where the display modeis a 2D display mode.

When the current display mode is a 3D display mode, and there is noplaylist being played back (e.g. when only the interactive graphics isdisplayed or the interactive graphics and the background data arecomposited and displayed), or processing to be performed is playback ofthe 3D playlist, the display mode is set as the 3D display mode.

When the current display mode is a 3D display mode, and there is noplaylist being played back (e.g. when only the interactive graphics isdisplayed or the interactive graphics and the background data arecomposited and displayed), or the display mode of the processing to beperformed is set as the 3D display mode by the BD-J application, thedisplay mode is set to be the 3D display mode.

When the current display mode is a 2D or 3D display mode, and there isno playlist being played back, or the playlist is a 3D playlist, thedisplay mode is set as the 3D display mode if the display mode by theBD-J application of the processing to be performed is the 3D displaymode.

This concludes the description of switching between the display modeswhen the playlist being played back is changed in the titles.

The following describes the components of the middleware. In FIG. 23,the middleware 33 includes a plane control module 42.

If the BD-J application sets the stereo mode of the background plane 11in the display mode storage unit 29 to ON, the composition unit 15performs composition such that the data held in the background plane 11that is written for the left eye is used for the left view, and the dataheld in the background plane 11 that is written for the right eye isused for the right view. When the stereo mode of the background plane 11is set to OFF, the data held in the background plane 11 for the righteye and the left eye should be the same. In case a vertically long areais used as the background plane 11, the upper half of the area is usedfor the left view, and the lower half of the area is used for the rightview. Note that when the stereo mode of the background plane 11 is setto OFF in such a case, the writing performed in the lower half of thearea by the application does not affect the display.

The method for specifying the stereoscopic effect of the image plane 8and the interactive graphics plane 10 is performed by stereo modeproperties in the image plane 8 setting and the interactive graphicsplane 10 setting. When the stereo mode is set to ON by the stereo modeproperty, the composition unit 15 performs the composition such that thedata held in the image plane 8 and the data held in the interactivegraphics plane 10 that have been written for the left eye are used forthe left view, and the data held in the image plane 8 and the data heldin the interactive graphics plane 10 that have been written for theright eye are used for the right view, as with the background plane 11.When the stereo mode is set to OFF, each of the data held in the imageplane 8 for the left eye and the right eye; and the data held in theinteractive graphics plane 10 for the left eye and the right eye shouldbe the same. In case a vertically long area that combines left-view andright-view is used for the image plane 8 and/or the interactive graphicsplane 10, the composition unit 15 performs the composition such that theupper half of the area is used for the left view, and the lower half ofthe area is used for the right view. Note that when the stereo mode ofthe image plane 8 and the stereo mode of the interactive graphics plane10 are set to OFF in such a case, the writing performed in the lowerpart of the area by the application does not affect the display.

The plane control module 42 is composed of libraries for the planecontrol, and realizes the plane control by the API call from theplatform unit.

(API Used for the Setting of the Display Mode Storage Unit 29)

The display mode storage unit 29 described in the above exists in aposition closer to the physical layer in the software layer model of theplayback apparatus 200. Therefore, the contents in the display modestorage unit 29 can be changed through the application programminginterface. The following describes the setting of the display modestorage unit 29 by instructions from the application, with reference toFIG. 27A and FIG. 27B. The following describes the applicationprogramming interface provided to the application by the plane controlmodule 42.

FIG. 27A and FIG. 27B shows the API used for setting in the display modestorage unit 29.

FIG. 27A shows five methods used for the plane setting. That is, agetConfigTemplate, a setPreference method, a getBestConfigurationmethod, a getCurrentConfiguration method, a setCoherentConfigurationmethod, and a setConfiguration method.

The following describes the acquisition of the plane status.

The BD-J application specifies, for example, a plane to acquire as anargument, and call the getConfigTemplate method to acquire a templateexpressing the status of the plane as the argument. FIG. 27B shows fourtemplates that can be acquired by the method. These four templates showthe background plane 11 setting, the video plane 6 setting, the imageplane 8 setting, the interactive graphics plane 10 setting,respectively.

Although the present embodiment shows the structure in which the planeoffset cannot be set in the background plane 11 setting and the videoplane 6 setting, a structure in which the plane offset can be set ispossible. In order to simplify the description, the following describesa structure in which the plane offset cannot be set in the backgroundplane 11 setting and the video plane 6 setting.

The following describes a THREE_D setting first.

The BD-J application acquires information on a parameter that can be setfrom the acquired template, specifies, as an argument, a parameter thatcan be set if necessary, and call the setPreference method to set theparameter that has been set as the argument. For example, when theparameter can be set as THREE_D setting, the BD-J application specifiesthe parameter as the argument, and calls the setPreference method to setthe THREE_D setting to ON or OFF. Also, the result of whether or not theapplication has succeeded in the setting.

When the BD-J application acquires the current display setting, thegefCurrentConfiguration is used. The getCurrentConfiguration is an APIthat causes the BD-J application to acquire the display setting of eachplane (the background plane 11, the video plane 6, the image plane 8,the interactive graphics plane 10) such as display mode and the stereomode that are currently set in the display mode storage unit 29.

Also, the getBestConfiguration method gives the BD-J application thebest setting information of each plane (the background plane 11, thevideo plane 6, the image plane 8 and the interactive graphics plane 10)in the playback apparatus 200.

Also, when the setCoherentConfigurations method is called from the BD-Japplication, individual plane setting can be specified at once (thebackground plane 11, the video plane 6, the image plane 8 and theinteractive graphics plane 10).

For example, when THREE_D setting of each plane (the background plane11, the video plane 6, the image plane 8 and the interactive graphicsplane 10) is ON, the BD-J application specifies information for settingthe THREE_D setting ON as the argument, and calls thesetCoherentConfiguration method to set the THREE_D setting of each plane(the background plane 11, the video plane 6, the image plane 8 and theinteractive graphics plane 10) ON.

Also, when the BD-J application specifies the resolution as theargument, and calls the setCoherentConfiguration method, the resolutionof each plane (the background plane 11, the video plane 6, the imageplane 8 and the interactive graphics plane 10) can be the same.

Although a description is given of a structure in which the BD-Japplication specifies the THREE_D setting and the resolution asarguments, and calls the setCoherentConfigurations method, the BD-Japplication may specify a plurality of parameters as arguments at once,and call the setCoherentConfigurations method.

While the setCoherentConfigurations method can set the setting of eachplane at once, the setConfiguration method can change the setting of theplanes specified as the arguments.

The following describes the configuration setting.

If the BD-J application specifies the configuration as the argument withuse of the template changed by the setPreference method, and calls thesetConfiguration method, or calls the setCoherentConfigurations method,the configuration is reflected in the display mode storage unit 29.

The following describes in what case the display mode changes.

When the BD-J application changes the display mode, the setConfigurationmethod is used. However, as a restriction, the BD-J application is notallowed to switch the display mode during the playback of the 2Dplaylist.

The following describes in what case switching between the displaysmodes fail.

The request for switching between the display modes by the BD-Japplication fails in the following case.

1. In a case where some of the THREE_D settings of each plane set at thetime of calling the setCoherentConfiguration method is ON and some areOFF.

2. In a case where the BD-J application attempts to set the displaysetting of the THREE_D setting to ON during the playback of the 2Dplaylist.

3. In a case during when the THREE_D setting is OFF the stereo mode getsspecified to turn ON.

In the above-stated cases 1 to 3, the getBestConfiguration method fails.

Furthermore, the following structure may be possible. The BD-Japplication makes a display mode switching request for the 3D displaymode, and acquires, from the display device connected to the playbackapparatus 200, information showing whether the display device is capableof display the 3D graphics. When the information indicates that thedisplay device is not capable of displaying the 3D graphics, the displaydevice replies to the BD-J application that it is not possible toresponds to the request (i.e. processing for executing the requestfails).

This concludes the description of the API used for settings in thedisplay mode storage unit 29.

(Setting of the Plane Offset)

The shifting amount used when the plane shift engine 19 performs theshifting, needs to be calculated to pixel value prior to shiftingprocess. It is desirable to adopt a plane offset for the stereoscopicviewing embedded in the MVC (Multi View Codec) video stream as aparameter for calculating the shift amount. However, the parameter forcalculating the shift amount is not limited to this, and it may bepreferable that the parameter is provided to the plane shift engine 19through various information elements that the content provider providesto the playback apparatus 200 through the BD-ROM, depending on theusecase.

The following describes the processing for setting the plane offset withreference to FIG. 28.

FIG. 28 shows the structure of a part relating to the setting of theplane offset. A writing control unit in the offset setting unit 20 has afunction of updating the plane shift value used by the plane shiftengine 19 for the shifting in each plane. The following (AA) to (FF) areprocessing for writing the plane offset values for the plane shiftengine 19 by the writing control unit in the offset setting unit 20.

(AA) The writing control unit can update the plane offset in the planesetting in the display mode storage unit 29 by the method call of thesetOffsetValue by the BD-J application.

The above-stated offset can be obtained by the BD-3 application in thegetOffsetValue method.

When the BD-J application calls the API; and the plane offset isembedded in the BD-J application, the flexibility is high, but theability to change offset in real-time according to depth of the video isdifficult. When the plane shift engine 19 refers to the offset specifiedby the BD-J application, the writing control unit in the offset settingunit 20 writes the offset value in the display mode storage unit 29 tothe plane offset value in the plane shift engine 19. The plane shiftengine 19 automatically shifts the plane in the horizontal directionbased on the plane offset values at the time of the composition.

The following describes timing of the setting.

At any time after the application has been started, the startedapplication can call the API that changes the depth of data held in eachof the image plane 8 and the interactive graphics plane 10. This iswhether the video is stopped or not. In either case (video is played ornot at the time of API call), it is possible to ensure that the shiftamount for the right view is in synchronization with the shift amountfor the left view, by controlling timing of setting the plane offsetvalue (from the display mode storage unit 29) in the plane shift engine19.

Specifically, the writing control unit does not update the plane offsetvalue in the plane shift engine 19 at timing that the BD-J applicationcalls the setOffset ( ). It is checked whether or not the writingcontrol unit has updated the plane offset in the display mode storageunit 29 at a time when one frame worth of the left-view and one frameworth of the right-view data have been outputted. In accordance with theupdate, the writing control unit updates the offset value of the planeshift engine 19. Thus, it is possible to ensure that the shift amountfor the right-view is in synchronization with the shift amount for theleft view. When the shift amount for the right view is not insynchronization with the shift amount for the left view, the displayappears in a way that is not intended by the content creator, whichresults in providing the unpleasant outputted video to the viewer.

(BB) When the BD-ROM is loaded, or when virtual package is constructed,the writing control unit updates the plane offset value in the planeshift engine 19 read from the meta file (ZZZZZ.xml) stored in the METAdirectory specified by the BD-ROM or the virtual package.

(CC) When reading and decoding of the MVC video stream starts, thewriting control unit updates, to the plane offset value in the planeshift engine 19, the plane offset embedded in a header area of each ofthe PES packets that compose the MVC video stream. Desirably, whenoutput of one frame worth of the left-view data and one frame worth ofthe right-view data has completed, the writing control unit updates theoffset corresponding to the next frame to be processed to a plane offsetvalue in the plane shift engine 19.

In case the plane offset is embedded in the MVC video stream, sinceoffset can be set for each video frame, the plane offset can be shifteddynamically but the cost of authoring can become a burden.

(DD) When the reading and decoding of the transport stream starts, thewriting control unit updates the plane offset embedded in the headerarea of the transport stream to the plane offset value in the planeshift engine 19. Desirably, when output of one frame worth of theleft-view data and one frame worth of the right-view data has completed,the writing control unit updates the offset corresponding to the framebeing processed as a plane offset value in the plane shift engine 19.

In a case where the plane offset value is embedded in the stream, theoffset value may be shifted dynamically with the video. Therefore, thereal time property is high, but the cost of authoring can become aburden.

(EE) When the current playlist is determined, and the playlistinformation is loaded, the plane offset of the playlist information isset to a plane offset value in the plane shift engine 19. When theplaylist information is used for determining the offset, flexibility ishigh at the time of authoring. However, compared to the case where theoffset is embedded in the stream, it is not possible to shorten the timeinterval from a time point at which an offset is set to a time point atwhich the offset is updated. Therefore, the real-time property is alittle poor.

(FF) Receiving the user operation that changes the level of the depth ofimage plane 8 and the data held in the interactive graphics plane 10 bythe operation of the button attached to a remote control or a device(i.e. the depth is expressed by the three levels such as “distant”,“normal” and “close”, or depth is expressed by the numerical values suchas “how many cm” or “how many mm”), the UO detection module 26 updatesthe plane offset value in the plane shift engine 19 with use of the useroperation. This update increases or decreases the plane offset dependingon the number of times the right-arrow key of the remote control ispressed. Thus, the graphics can be displayed closer to the viewer ordistant from the viewer by changing the number of times the right arrowor the left arrow is pressed. This enhances the operational property.

The shift amount by which the image plane 8 and the interactive graphicsplane 10 are shifted is obtained by performing the calculation based onthe plane offset in the plane shift engine 19 after the above-statedprocessing is performed. The reason why the calculation processing isnecessary is that while the shift amount of the pixel data pieces ineach of the image plane 8 and the interactive graphics plane 10 isdefined by the number of pixels, the plane offset is often defined byunits different from units of pixels.

If the plane offset is embedded in the MVC video stream, the plane shiftengine 19 calculates the shift amount by which the coordinates of thepixel data pieces held in the graphics plane based on the shift amountshown by the plane offset that the offset setting unit 20 a have storedin the plane shift engine 19, when output of one frame worth of theleft-view data and one frame worth of the right-view data havecompleted. This is because the plane offset is possibly changed for eachframe in some cases when the plane offset is stored in the MVC videostream.

This concludes the description of the various cases where the offsetsetting unit 20 sets the plane offset. The following describes a valueprovided by the user operation or the application.

The value provided from the user operation or the application may not bethe actual shift amount in pixels, but is possibly an adjusted valuefrom the value set in the current-state plane shift engine 19. In suchcase, the calculation of the plane offset value needs to be executed.For example, when the right-arrow key is pressed three times or a value“3” of a numerical value key is inputted, the plane shift engine 19 addsthis value to the plane offset set in the apparatus, and calculates theplane offset based on the added value. When the value is a “+” value,the shift amount is decreased, and the graphics appears to be moredistant from the viewer, for example. When the value is a “−” value, theshift amount is increased, and the graphics appears to be closer to theviewer, for example.

The following describes a change in depth.

As described in the above, when graphics such as subtitles and GUIs areshifted along the horizontal axis, the depth is changed by changing ashift amount for the subtitles, and shift amount for the GUI along thehorizontal axis. For example, the closer the left-view subtitles and theright-view subtitles become in a predetermined direction, the closer thegraphics are displayed to the screen. The more distant the left-viewsubtitles and the right-view subtitles become from one another in anopposite direction, the more distant the graphics are displayed from thescreen. However, the relationship between the plane offset and thepop-out level is greatly affected by the number of inches of the TV andthe characteristic of the liquid crystal of the 3D glasses. By settingstated coefficients in the terminal in advance, a value obtained bymultiplying the plane offset by this coefficient can be used for theshifting in order to realize such an optical effect. Multiplying theplane offset by the coefficient in such a manner makes it possible toadjust the pop-out level of the stereoscopic video based on thecharacteristics of the TV, the playback apparatus 200 and the liquidcrystal of the 3D glasses.

The following describes the various methods for shifting the coordinatesof each of the pixel data pieces.

When the coordinates of each of the pixel data pieces is shifted in theleft direction in the image plane 8 and the interactive graphics plane10, shifted data obtained as a result of the shifting is held in each ofthe planes. At the timing when the data for the right view is playedback, it is possible to adopt a method of shifting the shifted data heldin the graphics plane in a direction opposite from the left direction bya shift amount twice as much the above-stated shift amount. In thiscase, it is not necessary to perform the image decoding twice.

This concludes the description of the internal structure of the playbackapparatus 200.

(Implementation of the Display Mode Setting Initial Display Setting Unit28)

The following describes the implementation of the display mode settinginitial display setting unit 28. During a time period in which one titleis selected, and a BD-J object corresponding to the title is valid inthe playback apparatus 200, a new playlist is played back in some casesif the application in operation calls the JMF player instance accordingto the user operation. When the new playlist is played back, it isnecessary to make the setting of the display mode again in the title.

The function of the display mode setting initial display setting unit 28is to support: the display mode setting in the title when the titlechanges; the display mode setting when the playlist changes in thetitle; and the display mode setting set when the application expresslycalls the API. Specifically, the display mode setting initial displaysetting unit 28 may be implemented by: creating a program that causesthe MPU to execute the processing procedures shown in FIGS. 29 and 30;and installing the program in the playback apparatus 200.

FIG. 29 shows a flowchart showing processing procedures for setting thedisplay mode when the title is changed. In this flowchart, Step S24,Step S25 and Step S27 are selectively executed in accordance with thejudgment results of Step S21, Step S22, Step S23 and Step S26.

Step S21 is a judgment of whether the auto start playlist exists or not,and Step S22 is a judgment of whether the immediately preceding displaymode is a 3D display mode. Step S23 is a judgment of whether or not theauto start playlist of the selected title is a 3D playlist with aresolution of 1920*1080 or a 3D playlist with a resolution of 1280*720.

When the auto start playlist does not exist, it is judged whether thedefault resolution of the BD-J object is HD3D_(—)1920×1080, orHD3D_(—)1280×720 in Step S26. If the judgment of Step S26 is Yes, thedisplay mode is set to a 3D display mode, and the resolution is set to1920×1080 or 1280×720 in Step S25.

If the judgment of Step S26 is No, the display mode is set to a 2Ddisplay mode in Step S27, and the resolution is set to a defaultresolution in the BD-J object.

When the auto start playlist does exists, it is judged whether or notthe immediately preceding display mode is the 2D display mode in StepS22. If Step S22 is Yes, it is judged whether or not the playlist is a3D playlist in Step S23 and the resolution is 1920*1080 and 1280*720 inStep S23. If the judgment of Step S22 or Step 23 is No, the display modeis set to the 2D display mode, and a resolution is set to the resolutionof the auto start playlist in Step S24.

If the judgment of Step S22 is Yes, and the judgment of Step S23 is alsoYes, the display mode is set to a 3D display mode, and the resolution isset to 1920*1080 and 1280*720 according to the resolution of the autostart playlist in Step S25.

This concludes the description of processing procedures for setting thedisplay modes in the title.

FIG. 30 is a flowchart showing the processing procedures for setting thedisplay mode in the title. In this flowchart, Step S31 and Step S32 areserially connected to one another. Firstly, the setting of the displaymodes in the title starts with making request for playing back theplaylist. When receiving the request for playing back the playlists,Step S31 judges whether the current status is 3D or not. If the statusis 2D, the display mode is set to a 2D display mode in Step S34. StepS32 is a judgment of whether or not the playlist requested to be playedback is a 3D playlist. If the playlist is the 3D playlist, Step S33 isperformed, and the display mode is set to the 3D display mode. If theplaylist is the 2D playlist, Step S34 is performed, and the display modeis set to the 2D display mode.

This concludes the description of the processing procedures for settingthe display mode in the title.

(Implementation of the Playback Control Engine 14)

When a current playlist is selected due to some factor, the playbackcontrol engine 14 plays back the current playlist. Specifically, theplayback control engine 14 needs to realize processing of: readingplaylist information corresponding to the current playlist in the staticscenario memory 13; and playing back a 3D stream and a 2D streamreferred to by the playitem information in the playlist information.More specifically, it is necessary to: create a program that executesprocessing procedures shown in FIG. 32 and FIG. 33; install the programin the playback apparatus 200; and cause the MPU to execute the program.

FIG. 32 is a flowchart showing main procedures for playing back theplaylist in the BD-J mode.

Step S40 is a judgment of whether the current playlist number is set bysetting of the auto start playlist in the BD-J object that relates tothe selected title, or generation of the JMF player instance. If thecurrent playlist number is set, a playlist information file indicated bythe current playlist number is loaded in the scenario memory in StepS41. If a plane offset exists in the playlist information in Step S42,the offset setting unit 20 sets the plane offset as the plane offsetvalue in the plane shift engine 19. Then, the display mode in the titleis set in Step S43.

In Step S44, the first playitem number in the loaded playlistinformation is set to the current playitem number. In Step S45, thecurrent stream is selected from among PES streams permitted to be playedback in the current playlist information.

In Step S46, what number of a stream is used is decided based on theplayitem information.

In Step S47, it is judged whether the display mode determined in theStep S43 is the 2D display mode or the 3D display mode. If the displaymode is the 3D display mode, the playback control engine 14 executes theplayback of the 3D video stream in the 3D display mode in Step S49. Ifthe display mode is the 2D display mode, Step S48 is performed.

Step S48 is judgments of whether the video stream indicated by thecurrent stream number and whether the subtitles are 2D or 3D. If it isjudged that the video stream and the subtitles are 2D in Step S48, theplayback control engine 14 executes the playback of a 2D AV stream inthe 2D display mode in Step S51. If it is judged that the video streamand the subtitles are 3D, the playback control engine 14 executes theplayback of the 3D video stream in the 2D display mode in Step S50.Lastly, when the procedure reaches “end” in FIG. 32, the playback of theplaylist starts.

FIG. 33 is a flowchart showing playback procedures based on the playiteminformation.

In Step S60, the plane offset embedded in the video stream is set in theplane shift engine 19. In Step S61, a current PlayItem. In_Time and acurrent PlayItem. Out_Time are converted into a Start_SPN [i] andEnd_SPN [i] respectively, with use of an entry map corresponding to thea packet ID of the base view stream.

The SubPlatItemIn_Time and SubPlayItemOut_Time specified with use of anentry map [j] corresponding to a packet ID [j] of the enhanced viewstream are converted into Start_SPN [j] and End_SPN [j], respectively(Step S62).

The playback control engine 14 specifies an extent that falls in a readrange [i] for reading Start_SPN [i] to End_SPN [i] of a TS packet [i] ofthe packet ID [i] (Step S63). The playback control engibe 14 specifiesan extent that belongs to a read range for reading Start_SPN[j] toEnd_SPN [j] of a TS packet [j] of the packet ID [j] (Step S64). Theplayback control engine 14 instructs the drive to sorts addresses of theextents that belong to the read ranges [i] and [j] in an ascending orderin Step S65, and to sequentially read these extents that belong to theread ranges [i] and [j] with use of the sorted addresses in Step S66.

This concludes the description of the playback control engine 14.

(Implementation of the Plane Shift Engine 19)

The plane shift engine 19 realizes, in each display period for a videoframe, the processing of: reading the graphics data from each of theimage plane 8 and the interactive graphics plane 10; and compositing thegraphics data with picture data (sometimes called a video frame) to beplayed back in each of the display periods. The plane shift engine 19should support: reading processing and composition processing when thegraphics data is to be read from the image plane 8; and readingprocessing and composition processing when the graphics data is to beread from the interactive graphics plane 10. Furthermore, a type ofprocessing is determined depending on whether the stereo mode is ON orOFF. Also, a type of processing for the left view and a type ofprocessing for the right view are different. Therefore, in order toimplement the plane shift engine 19, it is necessary to: create aprogram that executes the processing procedures shown in FIG. 34 andFIG. 35 according to the display mode setting in the playback apparatus200; install the program in the playback apparatus 200; and cause theMPU to execute the program.

(Processing Procedures for the 3D Video Stream at the Time of the 3DDisplay Mode)

When the current display mode is the 3D display mode, and the playbacktargets are the 3D playlist and the 3D stream, the processing proceduresshown in FIG. 34 and FIG. 35 are executed.

Each of FIG. 34 and FIG. 35 is a flowchart showing 3D stream processingin the 3D display mode.

Each of the flowchart in FIG. 34 and the flowchart in FIG. 35 has a loopconfiguration that repeats Step S701 to Step S809. Each of the flowchartin FIG. 34 and the flowchart in FIG. 35 ends under a condition that itis judged that a next frame does not exists in Step S809. Step S701 toStep S707 in FIG. 34 are process taken for the left view, and Step S801to Step S809 in FIG. 35 are process taken for the right view. Firstly,the composition unit 15 acquires background data written in theleft-view background plane for the left view. The left-view backgroundplane stores the background data that has been written through the stillimage decoder 27 b according to the rendering instruction by the ED-Japplication (Step S701).

Next, after the video decoder 5 a decodes the left-view video stream,and writes the decoded left-view video stream in the video plane 6 (towhich (L) is given in FIG. 5), the composition unit 15 acquires theleft-view video data written in the video plane 6 (Step S702).

Then, the composition unit 15 checks the flag of the stereo mode in theimage plane 8 settings in the display mode storage unit 29 in Step S703.When the stereo mode is OFF, the composition unit 15 uses the imagedecoder 7 a to decode the left-view image, and writes the decodedleft-view image in the image plane 8 (a code (L)), then plane shiftengine 19 performs the shift processing for the left view (Step S704 a).

When the stereo mode is ON, the composition unit 15 uses the imagedecoder 7 a to decode the left-view image, and writes the decodedleft-view image in the image plane 8 (to which the code (L) is given).When the stereo mode is ON, the plane shift engine 19 does not performthe shift processing for the left view on the decoded left-view image inthe image plane 8 (to which the code (L) is given). This is because,when the stereo mode is ON, the composition unit 15 writes, in the imageplane 8 (to which a (R) is given), the right-view image which is seenfrom the different angle from the left-view image (Step S704 b).

The image plane 8 to which the code (L) is given in Step S704 a or inStep S704 b holds therein the image data which is stored in the imagememory 7, and is decoded by the graphics decoder 8.

Next, the composition unit 15 checks a flag of the stereo mode in theinteractive graphics plane 10 setting in the mode storage unit 29 inStep S705. When the stereo mode is OFF, the BD-J application writes theleft-view interactive graphics 10 in the left-eye plane (to which thecode (L) is given in the interactive graphics plane 10 in FIG. 5) withuse of the rendering engine 21, and acquires the left-view interactivegraphics from the left-eye plane (to which the code (L) is given in theinteractive graphics plane 10 in FIG. 5). The plane shift engine 19performs shift processing for the left-eye on the acquired left-viewinteractive graphics (Step S706 a).

When the stereo mode is ON, the BD-J application writes the left-viewinteractive graphics in the left-eye plane (to which the code (L) isgiven in the interactive graphics plane 10 in FIG. 5) with use of therendering engine 21. After that, although the BD-J application acquiresleft-view interactive graphics from the left-eye plane (to which thecode (L) is given in the interactive graphics plane 10 in FIG. 5) fordisplay, the plane shift engine 19 does not perform the shift processingfor the left eye on the left-view interactive graphics (Step S706 b).

The left-view interactive graphics plane from which the left-viewinteractive graphics is acquired in Step S706 a and Step S706 b storestherein the data that has been written through the rendering engine 21according to the rendering instruction by the BD-J application in theBD-J mode.

In case HDMV mode, the left-view interactive graphics plane storestherein the decoding result of the graphics data extracted from agraphics stream instead of graphics data drawn by the BD-J application.

In Step S707, the composition unit 15 composites: the background datawritten in the background plane 11 to which the code (L) is given inStep S701; video data written in the video plane 6 to which the code (L)is given in Step S702; the subtitle data written in the image plane 8 towhich the code (L) is given in Step S704; and the GUI data written inthe interactive graphics plane 10 to which the code (L) is given in StepS706 in order. Then, the resultant composited data is outputted to thedisplay as left-view data. In this case, the stereo mode is OFF in StepS703 and Step S705, data obtained as a result of the shift processingperformed in the corresponding planes are used for composition. As thefinal process of Step S707, the flag in the left-right processingstorage unit is changed when the data is outputted to the display. Notethat each process in Step S701 to 707 is processed as process for theleft eye. Which eye to process is determined by referring to theleft-right processing storage unit 17.

Next, the processing for the right eye is performed after the processingfor the left eye in Step S701 to S707 is completed.

FIG. 35 is a flowchart for describing the processing for the right eye.The composition unit 15 checks the flag of the stereo mode in thebackground plane 11 setting in the display mode storage unit 29 in StepS801. When the stereo mode is OFF, the composition unit 15 writes theleft-view background data held in the background plane 11 to which thecode (R) is given, and acquires the background data from the backgroundplane 11 to which the code (R) is given (Step S802 a). When the stereomode is ON, the composition unit writes the right-view background dataheld in the background plane 11 to which the code (R) is given, andacquires the right-view background data held in the background plane 11to which the code (R) is given (Step S802 b).

Next, the composition unit 15 checks the flag of the stereo mode in thevideo plane 6 setting in the display mode storage unit 29 in Step S803.When the stereo mode is OFF, the composition unit 15 uses the videodecoder 5 a to decode the left-view video stream, writes the decodedimage in the video plane 6 (to which the code (R) is given in FIG. 5),and acquires the left-view video data from the video plane 6 (to whichthe code (R) is given in FIG. 5) (Step S804 a). When the stereo mode isON, the composition unit 15 uses the video decoder 5 b to decode theright-view video stream, writes the decoded right-view video stream inthe video plane 6 (to which the code (R) is given in FIG. 5), andacquires the right-view video data from the video plane 6 the videoplane 6 (to which the code (R) is given in FIG. 5) (Step S804 b).

Then, the composition unit checks the flag of the stereo mode in theimage plane 8 setting in the display mode storage unit 29 in Step S805.When the stereo mode is OFF, the composition unit 15 writes theleft-view image decoded by the image decoder 7 a in the image plane 8(to which the code (R) is given). After that, the plane shift engine 19performs processing for the right eye on the left-view image written inthe image plane 8 (to which the code (R) is given) (Step S806 a). Whenthe stereo mode is ON, the composition unit 15 writes, in the imageplane 8 (to which the code (R) is given), the right-view image decodedby the image decoder 7 a. However, When the stereo mode is ON, the planeshift engine 19 does not perform the shift processing (Step S806 b).

The image plane 8 from which the images are acquired in Step S806 a andStep S806 b stores therein the subtitle data that has been stored in theimage memory 7, and decoded by the image decoder 7 (image decoders 7 aor 7 b).

Next, the composition unit 15 checks the flag of the stereo mode in theinteractive graphics plane 10 setting in the display mode storage unit29 in Step S807. When the stereo mode is OFF, the right-eye plane (towhich the code (R) is given in the interactive graphics plane 10 in FIG.5) contains left-view interactive graphics written by the BD-Japplication using the rendering engine 21. Then the plane shift engine19 performs the shift processing for the right eye on the left-viewinteractive graphics written in the right-eye plane (to which the code(R) is given in the interactive graphics plane 10 in FIG. 5) (Step S808a).

When the stereo mode is ON, the BD-J application uses the renderingengine 21 to write the right-view interactive graphics in the right-eyeplane (to which the code (R) is given in the interactive graphics plane10 in FIG. 5). However, the plane shift engine 19 does not perform theshift processing on the right-view interactive graphics written in theright-eye plane (to which the code (R) is given in the interactivegraphics plane 10 in FIG. 5) (Step S808 b).

In Step S809, the composition unit 15 composites: the background datawritten in the background plane 11 (to which the code (R) is given) inStep S802; the video data written in the video plane 6 (to which thecode (R) is given) in Step S804; the image data written in the imageplane 8 (to which the code (R) is given) in Step S806; and the GUI datawritten in the interactive graphics plane 10 in Step S808 in order. Incase the stereo mode is OFF in Step S805 and Step S807, data obtained asa result of the shift processing performed in the corresponding planeare targeted for the composition. The resultant composited data isoutputted to the display as the right-view data. When the compositeddata is outputted to the display, the flag in the left-right processingstorage unit 17 is changed. Note that Step S801 to Step S809 isprocessed for the right eye. Whether or not the current process is forthe right eye is judged by referring to the left-right processingstorage unit 17.

As long as frames are continuously inputted, the above-stated processingis repeated (Step S810).

Note that when the method for acquiring the offset from the header areaof the AV stream is adopted in the terminal, and the update of eachframe is implemented, it is necessary to update an offset value in theplane shift engine 19 to a value corresponding to the next frame by theoffset setting unit 20 during Step S810.

This concludes the description of the 3D stream processing in the 3Ddisplay mode.

(Processing Procedure in the 2D Display Mode)

When the display mode is the 2D display mode, the AV stream to be playedback may be one of the left-view stream and the right-view stream thatcompose the 3D stream, or may be a 2D stream. A description is given,taking a case where the 2D stream is used as an example.

When the current display mode is the 2D display mode, processingprocedures in FIG. 36A and FIG. 36B are executed.

FIG. 36A is a flowchart showing processing for the 2D stream in the 2Ddisplay mode.

Firstly, in Step S901, the composition unit 15 acquires the backgrounddata written in the background plane 11. Next, the video decoder 5 adecodes the 2D video stream, and writes the decoded video data held inthe video plane 6 (to which the code (L) is given), and the compositionunit 15 acquires the video data (Step S902).

Then, in Step S903, the composition unit 15 uses the image decoder 7 ato decode the 2D subtitle stream, writes the decoded image data held inthe image plane 8 (to which the code (L) is given), and acquires theimage data.

Next, in Step S904, the BD-J application uses the rendering engine 21 towrite the 2D graphics in the left-eye plane (to which the code (L) isgiven in the interactive graphics plane 10 in FIG. 5). The compositionunit 15 acquires the interactive graphics data from the interactivegraphics plane 10 (to which the code (L) is given).

In Step S905, the composition unit 15 composites: data held in thebackground plane 11 obtained in Step S901, data obtained in the videoplane 6 in Step S902, data obtained in the image plane 8 in Step S903and data obtained in the interactive graphics plane 10 in Step S904 inorder, and outputs the resultant composited data to the display. As longas frames are continuously inputted, processing returns to S901 (StepS810).

FIG. 36B is a flowchart showing the 2D display processing with use ofone of the left-view stream and the right-view stream that compose the3D stream (in this case, left-view video stream is used) in the 2Ddisplay mode.

The flowchart of FIG. 36B is created based on FIG. 34, and the samereference numerals are given to the steps that are same as the steps inthe flowchart of FIG. 34. The outline of the processing is almost thesame, but is briefly described. Firstly, in Step S701, the compositionunit 15 acquires the background data from the background plane 11(expressed as a code (L)).

Next, the left-view video data is decoded by the video decoder 5 a inthe video plane 6 (to which the code (L) is given), then the compositionunit 15 acquires the left-view video data from the video plane 6 (towhich the code (L) is given) (Step S702).

Then, in Step S704 b, the left-view image data is decoded by the imagedecoder 7 a in the image plane 8 (to which the code (L) is given), thenthe composition unit 15 acquires the stated image data from the imageplane 8 (to which the code (L) is given).

Next, in Step S706 b, the composition unit 15 acquires the left-viewinteractive graphics data, which is written by BD-J application usingthe rendering engine 21, held in the interactive graphics plane 10 (towhich the code (L) is given).

In S707, the composition unit 15 composites: the data held in thebackground plane 11 obtained in Step S701; the data held in the videoplane 6 obtained in Step S702; the data held in the image plane 8obtained in Step S704; and the data held in the interactive graphicsplane 10 obtained in Step S706 in order, and outputs the resultantcomposited data to the display. As long as frames are continuouslyinputted, processing returns to Step S701 (Step S810).

This concludes the description of the processing procedures in the 2Ddisplay mode.

(Processing Procedures for Updating the Offset Updating by the BD-JApplication)

The following describes what causes a change in depth by the BD-Japplication.

The following describes, with reference to FIG. 37, processing in a casewhere the application started by the BD-J module calls the API thatchanges the depths of data held in the image plane 8 and data held inthe interactive graphics plane 10.

FIG. 37 is a flowchart showing processing procedures when the BD-Japplication gives an instruction for updating the offset. In theflowchart of FIG. 37, Step S1001, Step S1002 and Step S1003 are added toStep 701 to Step S810 shown in FIG. 34 and FIG. 35 for displayprocessing for the 3D stream in the 3D display mode.

Up on receiving the calls from the API for changing the depth of thegraphics plane, the BD-J module updates the plane offset of the planewhich is a target of the display mode storage unit 29 based on the planeoffset received in Step S1001.

In this case, if the target plane is the image plane 8, the BD-J moduleupdates the plane offset in the image plane 8 setting, and if the targetplane is the interactive graphics plane 10, the BD-J module updates theplane offset in the interactive graphics plane 10 setting.

Meanwhile, in Step S1002, the BD-J module checks whether the image plane8 setting in the display mode storage unit 29 or the plane offset in theinteractive graphics plane 10 setting is updated or not. If the planeoffset is updated in Step S1002 (Step S1002: Yes), the BD-J moduleupdates the plane offset in the plane shift engine 19 (Step S1003). Notethat if the image plane 8 setting is updated in Step S1003, the BD-Jmodule updates the plane offset value in the image plane 8 in the planeshift engine 19, and if the interactive graphics plane 10 setting isupdated in Step S1003, the BD-J module updates the plane offset value inthe interactive graphics plane 10 in the plane shift engine 19. If theplane offset is not updated (Step S1002: No), Step S1003 is notperformed, and Step S701 starts.

As described in the present embodiment, it is possible to adjust adistance of how close the subtitle graphics are to the viewer, and adistance of how far the subtitle graphics are to the viewer from thedisplay by moving coordinates of each of the pixel data pieces in thegraphics plane (e.g. shifting in a direction horizontal to the displayscreen).

Furthermore, even in a case where both the right-view graphics(right-view interactive graphics) and left-view graphics (left-viewinteractive graphics) are not recorded on the recording medium (i.e.only the 2D graphics and 2D interactive graphics are provided), andtherefore the subtitle graphics will normally would appear to be buriedin the stereoscopic video because the subtitle graphics can be displayedOnly in a position on the display screen, it is possible to display thesubtitles and GUI in a position (e.g. a position at which the objectappear to be closer to the viewer than a position at which the 3D videois displayed) that does not interfere with the depth position in whichthe video stereoscopically pops out, by storing the 2D graphics in (2Dinteractive graphics) in a corresponding plane (to which a code (L) or(R) is provided) and adjusting the offset. Thus, it is possible tocomposite the subtitles and the GUI, and display the composited datawhile preventing the unpleasant stereoscopic display of the video asmuch as possible. Therefore, the appearance of the stereoscopic videocan be kept pleasant and will reduce the eye strain. Since the depth ofthe subtitle graphics can be adjusted by the above-stated adjustment, itis not necessary to prepare the left-view graphics and the right-viewgraphics for the stereoscopic playback of the video. Therefore, even ina case where a BD-ROM whose capacity is limited, and only graphics datafor one eye can be recorded on the media, it is possible to provide theuser with preferable stereoscopic view.

For an authoring studio, it is possible to skip the generation processof one of the right-view graphics for the stereoscopic playback, whichreduces the man-hours of generating films.

Second Embodiment

In the first embodiment, the shifting of the data held in each plane isrealized by shifting content in each memory element in the plane memory.However, the present embodiment relates to the improvement that realizesthe above-stated shifting when the pixel data pieces stored in the planememory is read by the composition unit 15 in units of lines. In a pixeldata aggregate with a resolution of 1920*1080 or 1280*720, a line (i.e.a group of 1920 horizontal pixels or a group of 1280 horizontal pixels)is referred to as line data. The composition unit 15 composites linedata read from each of the video plane 6, the background plane 11, theimage plane 8 and the interactive graphics plane 10.

Each of FIG. 39A, FIG. 39B and FIG. 39C shows a structure of thecomposition unit 15. As shown in FIG. 39A, FIG. 39B and FIG. 39C, thecomposition unit 15 is composed of line memories 50 to 53, an α1multiplication unit 54, a (1−α1) multiplication unit 55, a blend unit56, a (1−α2) multiplication unit 57, a scalar 58, α2 multiplication unit59, a blend unit 60, a scalar 61, an α3 multiplication unit 62, a (1−α3)multiplication unit 63, and a blend unit 64.

The line memory 50 stores therein line data read from the interactivegraphics plane 10.

The line memory 51 stores therein line data read from the image plane 8.

The line memory 52 stores therein line data read from the video plane 6.

The line memory 53 stores therein line data read from the backgroundplane 11.

The α1 multiplication unit 54 multiplies, by a transmittance α1,brightness of the line data composing pictures stored in the line memory51.

The (1−α1) multiplication unit 55 multiplies, by a transmittance (1−α1),brightness of the line data composing the graphics data stored in theline memory 53.

The blend unit 56 blends the line data whose pixels each is multipliedby the transmittance α1 by the α1 multiplication unit 54 with the linedata whose pixels each is multiplied by the transmittance (1−α1) by the(1−α1) multiplication unit 55.

The (1−α2) multiplication unit 57 multiplies the output from the blendunit 56 by the transmittance (1−α2).

The scalar 58 enlarges the line data read from the line memory 52.

The α2 multiplication unit 59 multiplies, by the transmittance α2,brightness of the line data composing the picture enlarged by the scalar58.

The blend unit 60 blends the line data multiplied by the transmittanceα2 with the line data whose pixels each is multiplied by thetransmittance (1−α2) by the (1−α2) multiplication unit 57.

The scalar 61 enlarges the line data read from the line memory 50.

The α3 multiplication unit 62 multiplies, by the transmittance α3,brightness of the line data that is read by the line memory 50, andcomposes the graphics enlarged by the scalar 61.

The (1−α3) multiplication unit 63 multiplies, by the transmittance(1−α3) brightness of the line data that is an output result obtainedfrom the blend unit 60.

The blend unit 64 blends the line data pieces each multiplied by thetransmittance α3 with the line data whose pixels each is multiplied bythe transmittance (1−α3) by the (1−α3) multiplication unit 63.

This concludes the description of the structure of the composition unit15.

The plane shift engine 19 described in the first embodiment realizes thetransfer processing of: reading pixel data pieces in units of lines frommemory elements in the background graphics plane 11, the video plane 6,the image plane 8 and the interactive graphics plane 10, and storing thepixel data pieces in the line memories 50 to 53. In this transferprocessing, it is possible to realize processing that is equivalent tothe shifting of coordinates of each of the pixel data pieces in thememory elements as described in the first embodiment by changing theaddresses of the line memories 50 to 53 to which the pixel data piecesare transferred.

FIG. 39A, FIG. 39B and FIG. 39C show processes for executing theshifting of coordinates of pixel data pieces composing each line data inthe plane in the right direction. FIG. 39A shows pixel data pieces ony^(th) line that exist in the graphics plane. FIG. 39B shows pixel datapieces on y^(th) line stored in the line memories.

As shown in FIG. 39B, if the plane offset is F pixels, it can be seenthat pixel data pieces at coordinates (F+1, y) to (1919, 0) on y^(th)line are stored. Also, it can be seen that empty space corresponding toF pixels exists at the right end of a memory element string in the linememory.

FIG. 39C shows that transparent pixel data pieces are added. It can beseen that the transparent pixel data pieces are added in the empty spacecorresponding to F pixels.

FIG. 40A, FIG. 40B and FIG. 40C shows processes for executing theshifting of coordinates of pixel data pieces composing each line data inthe plane in the left direction. FIG. 40A shows pixel data pieces ony^(th) line that exist in the graphics plane. FIG. 40B shows pixel datapieces on y^(th) line stored in the line memory. As shown in FIG. 40B,if the plane offset is F pixels, it can be seen that pixel data pieceson y^(th) line at coordinates (0, y) to (1919-F, 0) are stored. Also, itcan be seen that empty space corresponding to F pixels exists on theleft end of a memory element string in the line memory.

FIG. 40C shows that transparent pixel data pieces are added. It can beseen that the transparent pixel data pieces are added in the empty spacecorresponding to F pixels.

This concludes the description of processes for executing the shiftingof coordinates of pixel data pieces composing each line data in theplane.

The plane shift engine 19 that executes the shifting of coordinates ofpixel data pieces composing each line data in the plane can beimplemented by creating a program that causes the MPU to execute theprocessing procedures shown in a flowchart of FIG. 41, and installingthe program in the playback apparatus 200. The following describes theimplementation of the plane shift engine 19 by software.

FIG. 41 is a flowchart showing processing procedures for storing pixeldata pieces in the line memory.

Processing from Step S112 to Step S115 has a loop configuration thatrepeats Step S112 to Step S115 after a parameter i is set to 0 in StepS111. This loop ends under a condition that the parameter i becomes avalue (the number of lines−1) in Step S114. As long as it is judged thatthe parameter i is not a value (the number of lines−1) in Step S114, theparameter i is incremented, and Step S112 to Step S115 are repeated. InStep S112, the plane shift engine 19 reads pixel data pieces (Xo, Yi) to(Xn−plane offset−1, Yi) in the graphics plane, and writes the pixel datapieces in (Xo+plane offset) to (Xn). In Step S113, the plane shiftengine 19 writes transparent pixel data pieces in Xo to Xo+offset−1 inthe line memory.

Processing from Step S121 to Step S125 has a loop configuration thatrepeats Step S121 to Step S125 after a parameter i is set to 0 in StepS121. This loop ends under a condition that the parameter i becomes avalue (the number of lines−1) in Step S124. As long as it is judged thatthe parameter i is not a value (the number of lines−1) in Step S124, theparameter i is incremented, and Step S122 to Step S125 are repeated. InStep S122, the plane shift engine 19 reads pixel data pieces (Xn+planeoffset, Yi) to (Xn, Yi) in the graphics plane, and writes the pixel datapieces in (Xo) to (Xn−plane offset−1). In Step S123, the plane shiftengine 19 writes transparent pixel data pieces in (Xo) to (Xo+offset−1)in the line memory.

This concludes the description of the processes for executing theshifting of coordinates of pixel data pieces composing each line data inthe plane in the left direction.

According to the present embodiment as described in the above, thememory access to the graphics plane can be less frequent since theshifting of coordinates of pixel data held in the right direction andthe left direction is realized in each line memory when the plane shiftengine 19 reads the pixel data pieces in the graphics plane for eachline data.

(Notes)

This concludes the description of the best modes of carrying out theinvention known to the applicant at the time of application. However,further improvements and variations related to the technical topicsindicated below can be added. Whether to carry out the invention asindicated in the embodiments or to use these improvements and variationsis arbitrary, and is left to the discretion of the one who carries outthe invention.

(Implementation as the Recording Medium)

The playback apparatus 200 includes a local storage including a built-inmedia library and a removable media. Therefore, since the playbackapparatus 200 is expected to write data in the built-in media drive andthe local storage, the playback apparatus 200 described in the presentDescription has a function as the recording medium. When the playbackapparatus 200 functions as the recording medium, the playback apparatus200 executes the writing of a management object in the following twomanners (1) and (2).

(1) When the playback apparatus 200 has a function of playing back thevirtual package, the playback apparatus 200 writes the BD-J object inthe following way. That is, when the BD-ROM is loaded, the playbackapparatus 200 acquires additional contents corresponding to the BD-ROMfrom a WWW server via the network in accordance with a request from anapplication. The acquired additional contents include the BD-J object inwhich the GUI management table is written. A control unit, in theplayback apparatus 200, that perform recording control writes theacquire BD-J object in the local storage in accordance with the requestfrom the application. Thus, it is possible to configure the virtualpackage by compositing the contents in the BD-ROM with the additionalcontents recorded in the local storage.

In the BD-ROM are recorded an identifier of a disc root certificate, anidentifier of an organization that distributes the BD-Rom contents andan identifier of the BD-ROM. An area in which the additional contentsare stored are specified by a file path including the identifier of thedisc root certificate, the organization identifier and the BD-ROMidentifier.

The application performs writing by sending, to the control unit, thefile path that specifies the area in which the additional contents arestored.

When the local storage has a file system that limits the names ofdirectories and files to 255 characters or less, the file path used forwriting to the local storage includes file names and extensions in a8.3-format file system in which directory names are 8 characters or lessand file names and extension names are three characters or less.

(2) If the playback apparatus 200 has a function of receiving the BD-Jobject from an on-demand manufacture service or an electricalsell-through services (MODEST), the playback apparatus 200 writes theBD-J object as follows.

That is, when the playback apparatus 200 receives the BD-J object froman on-demand manufacture service or an electrical sell-through services(MODEST), the playback apparatus 200 creates a default directory and aMODEST directory under the root directory in the removable media, andcreates the BDMV directory under the MODEST directory. The MODESTdirectory is a first MODEST directory. The first MODEST directory iscreated when the user receives the service for the first time. When theuser receives the service for the second time onwards, the control unitin the playback apparatus 200 creates a MODEST directory correspondingto a service from the second time onwards.

As described in the above, acquiring the BD-J object in which the GUImanagement table is written, the control unit writes a start-up programin the default directory, and writes the BD-J object in the BDMVdirectory under the MODEST directory. This start-up program is a programto be executed first when the recording medium is loaded in the playbackapparatus 200. The start-up program causes the playback apparatus 200:to display a menu from which user operation of selecting the BDMVdirectory is received, and to execute a root change function. This rootchange function is a function of causing the MODEST directory to whichthe BDMV directory belongs to be recognized as the root directory whenthe user performs the selection operation on the menu. The root changefunction makes it possible to execute the playback control based on theacquired BD-J object by the same control procedures as the controlprocedures for playing back the BD-ROM.

(Java™ Application)

The BD-J application may be, for example, an Electronic Commerce (EC)client application, or may be an online game played against opponents onthe Internet. Furthermore, by working together with a search engine,various online services can be provided to the user.

(Units for Integrating the GUI Table)

Although in the embodiment, the GUI management object is provided in theBD-J object, the invention may be configured so as to provide the GUImanagement table to correlate PlayList information and PlayIteminformation, and so that the timing at which the current PlayListbecomes the current PlayItem or when the current PlayItem becomes thespecified PlayItem, the plane memory is released, and plane memories forstereoscopic playback and for planar playback may be reserved. Thisenables the area of the memory device to be managed with more precisetime accuracy.

(Video Stream for Stereoscopic View)

Recording left view and right view video streams on a BD-ROM is only oneexample. Playback may also be provided by recording on the BD-ROM, foreach picture, as an EnhancedView video stream, a video stream indicatingper pixel depth values.

(Package to be Implemented)

When implementing an playback apparatus, the following BD-J Extensionsare preferably implemented in the playback apparatus. The BD-JExtensions include various packages that have been specialized forproviding functions exceeding GEM [1.0.2] to a Java™ format. Packages towhich BD-J Extensions are provided are as follows.

org.bluray.media

This package provides specialized functions to be added to the Java™Media FrameWork. Control for angles, audio, and subtitle selection areadded to the package.

org.bluray.ti

This package includes a structure for referring to an API for operatingby mapping “services” to “titles” according to GEM [1.0.2] and referringto title information from a BD-ROM, and a structure for selecting a newtitle.

org.bluray.application

This package includes an API for managing life cycles of applications.Also, this package includes an API for referring to necessaryinformation for signaling when executing applications.

org.bluray.ui

This package includes a class that defines a constant number for keyevents specialized in the BD-ROM, and realizes synchronization with thevideo playback.

org.bluray.vfs

This package provides a binding scheme for binding contents that arerecorded on the BD-ROM (on-disk contents) and contents in the LocalStorage that are not recorded on the BD-ROM (off-disk contents) andplaying back the contents seamlessly, regardless of the location of thecontents.

The Binding Scheme correlates contents on the BD-ROM (AV clips,subtitles, BD-J applications) with related contents in the LocalStorage. This Binding Scheme realizes seamless playback regardless ofthe location of the contents.

(Structure of the Graphics Plane)

When the image plane 8 and the interactive graphics plane 10 arecomposed of SDRAM, a unique control for shifting the coordinates of thepixel data pieces stored in the SDRAM is necessary.

The SCRAM includes a ROW address decoder that outputs a ROW addressoutput at time-divided address pins to a memory array, a COLUMN addressdecoder that outputs a COLUMN address output at time-divided addresspins to a memory array, a page data buffer that holds one page lengthworth of data read from the memory array, and outputs the one pagelength worth of data at the address pins. Since the SDRAM uses acondenser as a storage element, natural discharge occurs. For thisreason, it is necessary in the SDRAM to perform refresh on the storageelements, and additional circuits must be added for performing refresh.

When a picture is stored in the SDRAM, the plane controller forms acommand by compositing the states of RAS, CAS, WE, CS, and CKE, and bycompositing the command and a composition of the addresses, reading andwriting are performed. For example, when using a burst transfer mode, anactivated command is issued, and a ROW address is issued at the addresspin of the SDRAM. After a delay of a fixed time period, a READ commandis issued, and the COLUMN address is issued at the address pin.

(Programming Language Application Range)

It is described in the embodiments that Java™ language is used as thevirtual machine programming language. However, the language is notlimited to Java™, and other programming languages, such as B-Shell, PerlScript, and ECMA Script, which are used in UNIX™ OS and the like, mayalso be used.

(Changing to Multidrive)

The above-described embodiments describe a BD-ROM as an example of arecording medium, and a BD-ROM drive as an example of a specific devicefulfilling a function of reading data from the BD-ROM. However, a BD-ROMis merely one example, and it is also possible to perform the operationsdescribed in the above embodiments when an optical disk medium such asBD-R, BD-RE, DVD, or CD is used as the recording medium, data having theabove-described data structure is stored on such recording medium, andthere is a drive device capable of reading such recording medium.

The recording media of the embodiments include all types of packagemedia such as optical disks, semi-conductor memory cards, etc. Therecording media of the embodiments described, as an example, an opticaldisk (for example, a preexisting read-only optical disk such as a BD-ROMor a DVD-ROM). However, the present invention is not limited to this.For example, it is possible to implement the present invention bywriting 3D contents, including data that is necessary for implementingthe present invention and has been broadcast or distributed over anetwork, with use of a terminal device fulfilling a function of writing3D contents (for example, the function may be included in the playbackapparatus, or may be included in an apparatus other than the playbackapparatus), on a writable optical disk (for example, a preexistingwritable optical disk such as BD-RE or DVD-RAM).

Also, it is possible to implement the present invention when therecording medium is, besides an optical disk, for example, a removablemedium such as an SD memory card (semiconductor memory card).

When a semiconductor memory is used instead of the BD-ROM, data in theread buffer 1 and the read buffer 2 is transferred to the heap memory31, the dynamic scenario memory 23 and the static scenario memory 13 viaan interface (memory card I/F) for reading the data in the semiconductormemory.

More specifically, when the semiconductor memory card is inserted into aslot (not shown) in the playback apparatus 200, the playback apparatus200 and the semiconductor memory card are electrically connected to eachother via the memory card I/F. The data recorded in the semiconductormemory card is transferred to the read buffer 1, the read buffer 2, theheap memory 31, the dynamic scenario memory 23 and the static scenariomemory 13 via the memory card I/F.

From a standpoint, for example, of improving the confidentiality of dataand copyright protection, there are cases in which portions of the datarecorded on the BD-ROM are encoded as necessary.

For example, the encoded data of the data recorded on the BD-ROM may be,for example, data corresponding to a video stream, data corresponding toan audio stream, or data corresponding to a stream that includes bothvideo and audio.

The following describes deciphering of encoded data that is among thedata recorded on the BD-ROM.

In the playback apparatus, data corresponding to a key necessary fordeciphering encoded data on the BD-ROM (for example, a device key) isrecorded in the playback apparatus in advance.

Meanwhile, data corresponding to the key necessary for decipheringencoded data (for example, an MKB (media key block) corresponding to thedevice key) and data in which the key itself, for deciphering theencoded data, is encoded (for example an encoded title key correspondingto the device key and the MKB), is recorded on the BD-ROM. Here, thedevice key, the MKB, and the encoded title key correspond to each other,and furthermore correspond to an identifier (for example, a volume ID)written in an area that cannot be normally copied on the BD-ROM (an areacalled BCA). If this composition is not correct, the code cannot bedeciphered. Only if the composition is correct, the key necessary fordeciphering the code (for example, a decoded title key obtained bydecoding the encoded title key based on the device key, the MKB andvolume key, can be elicited, and with use of the key necessary for theencoding, the encoded data can be deciphered.

When the inserted BD-ROM is played back in the playback apparatus,encoded data cannot be played back unless the BD-ROM includes a devicekey that is paired with a title key or MKB (or corresponds to a titlekey or MKB). The reason is that the key necessary for deciphering theencoded data (the title key) is itself encoded when recorded on theBD-ROM (as an encoded title key), and if the composition of the MKB andthe device key is not correct, the key necessary for deciphering thecode cannot be elicited.

On the other hand, the playback apparatus is configured so that, if thecomposition of the encoded title key, MKB, device key, and volume ID iscorrect, the video stream is decoded, for example with use of the keynecessary for deciphering the code (the decoded title key obtained bydecoding the encoded title key based on the device key, the MKB and thevolume ID), and the audio stream is decoded by the audio decoder.

Although in the present embodiments, a BD-ROM is described as an exampleof a recording medium, the recording medium is not limited to being aBD-ROM, and the present invention can be implemented even when using,for example, a readable/writable semiconductor memory (for example, asemiconductor memory card having a nonvolatile property such as an SDcard).

For example, the playback apparatus may be configured to record datacorresponding to data recorded on the BD-ROM on a memory card with useof digital distribution, and to play back the data from thesemiconductor memory card. When distributing the necessary data with useof digital distribution and recording the distributed data, it ispreferable to distribute the data after having performed partial orentire decoding of the distributed data as necessary, and leaving datathat is necessary for the semiconductor memory card in an encoded state.

The following describes operation using, for example, digitaldistribution, for recording data (distributed data) corresponding to thedata described in the above embodiments on the semiconductor memory.

The operations described above may be configured to be performed by theplayback apparatus described in the embodiments, or by a terminalapparatus dedicated to recording distributed data on a semiconductormemory that is separate from the playback apparatus in the embodiments.Here, an example of the playback apparatus performing the operations isdescribed. Also, an SD card is described as an example of the recordingdestination.

When recording distributed data to the SD memory card inserted in theslot of the playback apparatus, first, transmission is requested of thedistributed data to a distribution server (not illustrated) thataccumulates the distributed data. The playback apparatus reads, from theSD memory card, information for uniquely identifying the SD memory cardthat is inserted at the playback apparatus at this time (for example, aspecific identification number assigned individually to the particularSD memory card, more specifically, a serial number of the SD memorycard, etc.), and transmits the read identification information to thedistribution server along with the distribution request.

This identification information for uniquely identifying the SD memorycard corresponds to, for example, the above-described volume ID.

Meanwhile, in the distribution server, decoding is performed so thatnecessary data among the data that is distributed (video streams, audiostreams, etc.) can be deciphered with use of the key that is necessaryfor deciphering the code (for example, the title key), and the necessarydata is stored in the server.

For example, a private key is stored in the distribution server, and thedistribution server is configured so that different public keys aredynamically created to correspond respectively to the semiconductormemory card-specific identification numbers.

Also, the distribution server is configured so that encoding is possibletowards the key that is necessary for deciphering the encoded dataitself (the title key) (in other words, configured so that an encodedtitle key can be generated).

The generated public key information includes information correspondingto the above-described MKB, volume ID, and encoded title key. If thecomposition of, for example, the semiconductor memory card-specificidentification number, the actual public key included in the public keyinformation described later, and the device key recorded in advance inthe recording apparatus, is correct, the key necessary for decipheringthe code (for example, the title key obtained by decoding the encodedtitle key based on, for example, the device key, the MKB, and thesemiconductor memory card-specific identification number) is acquired,and with use of this acquired key (title key) necessary for decipheringthe code, decoding of the encoded data can be performed.

Next, the playback apparatus records the received public key informationand distributed data in the recording area of the semiconductor memorycard inserted in the slot.

Next, the following describes an exemplary method for decoding andplaying back encoded data, from among the data included in the publickey information recorded in the recording area of the semiconductormemory card and the data included in the distribution data.

The received public key information is, for example, recorded on adevice list indicating the public key itself (for example, the MKB andthe encoded title key), signature information, the semiconductor memorycard-specific identification number, and information pertaining to adevice to be invalidated.

The signature information includes, for example, hash values of thepublic key information.

In the device list, information is recorded pertaining to, for example,an apparatus that is possibly performing unauthorized playback. This isfor example a device key, an identification number of the playbackapparatus, or an identification number of a decoder in the playbackapparatus recorded in advance on the playback apparatus, information foruniquely specifying the device, a part included in the apparatus, or afunction (program) of the apparatus possibly performing unauthorizedplayback.

The following description pertains to playback of encoded data fromamong the distribution data recorded in the recording area of thesemiconductor memory card.

First, a check is performed pertaining to whether the encoded key itselfmay be operated before decoding the encoded data with use of the publickey itself.

Specifically, the following checks are performed:

(1) whether there is a match between the semiconductor memoryidentification information included in the public key information andthe specific identification number stored in advance on thesemiconductor memory card,

(2) whether there is a match between a hash value of the public keyinformation calculated in the playback apparatus, and a hash valueincluded in the signature information, and

(3) whether, based on information indicated in the device list includedin the public key information, the playback apparatus performingplayback is possibly performing unauthorized playback (for example, bychecking whether the device key included in the device list matches thedevice key stored in the playback apparatus in advance). These checksmay be performed in any order.

Control is performed so that the playback apparatus does not decode theencoded data if any of the following is satisfied, in theabove-described checks 1 to 3: i) the semiconductor memory-specificidentification information included in the public key information doesnot match the specific identification number stored in advance on thesemiconductor memory card, ii) the hash value of the public keyinformation calculated in the playback apparatus does not match the hashvalue included in the signature information, or iii) a judgment is madethat the playback apparatus performing the playback is possiblyperforming unauthorized playback.

Also, a judgment is made that the composition of the semiconductormemory-specific identification number, the public key included in thepublic key information, and the device key recorded in advance in theplayback apparatus, is correct if i) the semiconductor memorycard-specific identification information included in the public keyinformation matches the specific identification number saved on thesemiconductor memory card in advance, (ii), the hash value of the publickey information calculated in the playback apparatus matches the hashvalue included in the signature information, and (iii) a judgment ismade that the playback apparatus is not possibly performing unauthorizedplayback. When the composition is judged to be correct, the encoded datais deciphered with use of the key necessary for deciphering the code(based on a device key, the MKB and the semiconductor memory-specificidentification number).

For example, when the encoded data is a video stream and an audiostream, the video decoder decodes (decodes) the video stream with use ofthe key necessary for deciphering the code (the title key obtained bydecoding the encoded title key), and the audio decoder decodes (decodes)the audio stream with use of the key necessary for deciphering the code.

According to this type of structure, for any playback apparatus, parts,function (program), etc. that is possibly performing unauthorized use atthe time of electronic distribution, information for the identificationof such is provided to the device list, and if distribution isattempted, since playback decoding with use of public key information(the public key) can be suppressed on the playback apparatus side ifinformation is included that is indicated in the device list, even ifthe composition of the semiconductor memory-specific identificationnumber, the public key included in the public key information, and thedevice key recorded in the playback apparatus in advance, is correct,since control can be performed so that the deciphering of the encodeddata is not performed, use of distributed data on an unauthorized devicecan be suppressed.

Also, it is preferable to use a structure in which the semiconductormemory card-specific identifier recorded in advance on the semiconductormemory card is stored in a highly confidential recording area. Thereason is that when the specific number recorded on the semiconductormemory card (for example, in the example of an SD memory card, the SDmemory card serial number, etc.) has been altered, illegal copying isfacilitated. The reason is that different specific identificationnumbers are allocated to different semiconductor memory cards, but ifthe specific identification numbers are altered to be the same, thejudgment in (1) becomes meaningless, and there is a possibility ofillegal copying, corresponding to the number that was altered, beingperformed.

Accordingly, it is preferable for the information that is thesemiconductor memory card-specific identification number to be recordedin a high-confidentiality recording area.

To realize this type of structure, for example by providing a recordingarea (called a second recording area) that is separate from therecording area (called a first recording area) that stores normal dataas recording areas for recording high-confidentiality data that issemiconductor memory card-specific identifiers), and providing a controlcircuit for accessing the recording area, access to the second recordingarea can be made only via the control circuit.

For example, the data recorded in the second recording area has beenencoded and recorded. For example, a circuit for decoding the encodeddata is built into the control circuit. When there is access of the datain the second recording area via the control circuit, the structure needmerely be such that the code is decoding and the decoded data isreturned. Also, if the control circuit stores information of a storagelocation of data recorded in the second recording area, and there is arequest to access the data, the corresponding storage location of thedata need merely be specified, and the data read from the specifiedstorage location be returned.

Upon issuing an access request to the data recorded in the secondrecording area to the control circuit via the memory card I/F (forexample, semiconductor memory-specific identification number),applications that operate in the playback apparatus that requestrecording to the semiconductor memory card with use of digitaldistribution, the control circuit that received the request reads thedata recorded in the second recording area and returns the data to theapplication operating in the playback apparatus. Along with thesemiconductor memory card-specific identification number, thedistribution request for the necessary data need only be requested fromthe distribution server, and the public key information sent from thedistribution server and the corresponding request for distribution ofthe data, may be recorded to the first recording area.

Also, an application operating in the playback apparatus, that requestsrecording on the semiconductor memory card with use of digitaldistribution, before issuing the request to the control circuit via thememory card I/F to access the data recorded on the second recording area(for example, the semiconductor memory card-specific identificationnumbers), preferably checks in advance whether the application has beenaltered. For example, a digital certificate compliant with preexistingX.509 specifications may be used in the check for alteration.

Also, access to the distribution data recorded in the first recordingarea of the semiconductor memory card need not necessarily be access viaa control circuit on the semiconductor memory card.

(Implementation as a Program)

The application program described in the embodiments can be made asdescribed below. First, the software developer, with use of aprogramming language, writes a source program to realize the content ofthe flowcharts and the functional structural elements. When writing thesource program that embodies the content of the flowcharts and thefunctional structural elements, the software developer uses the classstructures, variables, array variables, and external function calls towrite the program in accordance with the syntax of the programminglanguage.

The written source programs are given as files to a compiler. Thecompiler translates the source programs and creates an object program.

The translation by the compiler is made up of the processes of syntaxanalysis, optimization, resource allocation, and code generation. Syntaxanalysis involves performing lexical analysis and semantic analysis ofthe source programs, and converting the source programs to anintermediary program. Optimization involves performing operations todivide the intermediary program into basic blocks, analyze the controlflow of the intermediary program, and analyze the data flow of theintermediary program. In resource allocation, to improve suitabilitywith a command set of a targeted processor, variables in theintermediary program are allocated to a register or a memory in atargeted processor. Code generation is performed by converting theintermediary commands in the intermediary program into program code, andobtaining an object program.

The object program generated here is made up of one or more programcodes for executing, on a computer, the steps of the flowcharts and thevarious processes carried out by the functional structural elements inthe embodiments. Here, program code may be any of various types such asnative code of a processor or JAVA byte code. There are various formatsfor realization of the steps by the program code. If it is possible touse external functions to realize the steps, call texts that call suchfunctions become program code. Also, there are cases in which a programcode for realizing one step is attributed to separate object programs.In a RISC processor in which command types are limited; the steps of theflowcharts may be realized by compositing calculation operationcommands, logical calculation commands, branch instruction commands,etc.

When the object programs have been created, the programmer starts up alinker. The linker allocates the object programs and library programs tomemory spaces, composites the object programs and library programs intoone, and generates a load module. The load module generated thus isanticipated to be read by a computer, and causes the computer to executethe processing procedures and functional structural components shown inthe flowcharts. The programs may be provided to users by being recordedon a recording medium that is readable by a computer.

(Singular Implementation of the LSI)

The system LSI is obtained by implementing a bare chip on a high-densitysubstrate and packaging them. The system LSI is also obtained byimplementing a plurality of bare chips on a high-density substrate andpackaging them, so that the plurality of bare chips have an outerappearance of one LSI (such a system LSI is called a multi-chip module).

The system LSI has a QFP (Quad Planar view Package) type and a PGA (PinGrid Array) type. In the QFP-type system LSI, pins are attached to thefour sides of the package. In the PGA-type system LSI, a lot of pins areattached to the entire bottom.

These pins function as an interface with other circuits. The system LSI,which is connected with other circuits through such pins as aninterface, plays a role as the core of the playback apparatus 101.

Such a system LSI can be embedded into various types of devices that canplay back images, such as a television, game machine, personal computer,one-segment mobile phone, as well as into the playback apparatus 102.The system LSI thus greatly broadens the use of the present invention.

When an elementary buffer, video decoder, audio decoder, and graphicsdecoder are integrated into a system LSI, it is desirable that thesystem LSI conforms to the Uniphier architecture.

A system LSI conforming to the Uniphier architecture includes thefollowing circuit blocks.

Data Parallel Processor (DPP)

The DPP is an SIMD-type processor where a plurality of elementalprocessors perform a same operation. The DPP achieves a paralleldecoding of a plurality of pixels constituting a picture by causingoperating units, respectively embedded in the elemental processors, tooperate simultaneously by one instruction.

Instruction Parallel Processor (IPP)

The IPP includes: a local memory controller that is composed ofinstruction RAM, instruction cache, data RAM, and data cache; processingunit that is composed of instruction fetch unit, decoder, executionunit, and register file; and virtual multi processing unit that causesthe processing unit to execute a parallel execution of a plurality ofapplications.

MPU Block

The MPU block is composed of: peripheral circuits such as ARM core,external bus interface (Bus Control Unit: BCU), DMA controller, timer,vector interrupt controller; and peripheral interfaces such as UART,GPIO (General Purpose Input Output), and sync serial interface.

Stream I/O Block

The stream I/O block performs data input/output with the drive device,hard disk drive device, and SD memory card drive device which areconnected onto the external busses via the USB interface and the ATApacket interface.

AV I/O Block

The AV I/O block, which is composed of audio input/output, videoinput/output, and OSD controller, performs data input/output with thetelevision and the AV amplifier.

Memory Control Block

The memory control block performs reading and writing from/to the SD-RAMconnected therewith via the external buses. The memory control block iscomposed of internal bus connection unit for controlling internalconnection between blocks, access control unit for transferring datawith the SD-RAM connected to outside of the system LSI, and accessschedule unit for adjusting requests from the blocks to access theSD-RAM.

The following describes a detailed production procedure. First, acircuit diagram of a part to be the system LSI is drawn, based on thedrawings that show structures of the embodiments. And then theconstituent elements of the target structure are realized using circuitelements, ICs, or LSIs.

As the constituent elements are realized, buses connecting between thecircuit elements, ICs, or LSIs, peripheral circuits, interfaces withexternal entities and the like are defined. Further, the connectionlines, power lines, ground lines, clock signals and the like aredefined. For these definitions, the operation timings of the constituentelements are adjusted by taking into consideration the LSIspecifications, and bandwidths necessary for the constituent elementsare reserved. With other necessary adjustments, the circuit diagram iscompleted.

After the circuit diagram is completed, the implementation design isperformed. The implementation design is a work for creating a boardlayout by determining how to arrange the parts (circuit elements, ICs,LSIs) of the circuit and the connection lines onto the board.

After the implementation design is performed and the board layout iscreated, the results of the implementation design are converted into CAMdata, and the CAM data is output to equipment such as an NC (NumericalControl) machine tool. The NC machine tool performs the SoCimplementation or the SiP implementation. The SoC (System on Chip)implementation is a technology for printing a plurality of circuits ontoa chip. The SiP (System in Package) implementation is a technology forpackaging a plurality of circuits by resin or the like. Through theseprocesses, a system LSI of the present invention can be produced basedon the internal structure of the playback apparatus 200 described ineach embodiment above.

It should be noted here that the integrated circuit generated asdescribed above may be called IC, LSI, ultra LSI, super LSI or the like,depending on the level of the integration.

It is also possible to achieve the system LSI by using the FPGA (FieldProgrammable Gate Array). In this case, a lot of logic elements are tobe arranged lattice-like, and vertical and horizontal wires areconnected based on the input/output compositions described in LUT(Look-Up Table), so that the hardware structure described in eachembodiment can be realized. The LUT is stored in the SRAM. Since thecontents of the SRAM are erased when the power is off, when the FPGA isused, it is necessary to define the Config information so as to write,onto the SRAM, the LUT for realizing the hardware structure described ineach embodiment.

In the embodiment, the invention is realized by middleware and hardwarecorresponding to the system LSI, hardware other than the system LSI, aninterface portion corresponding to the middleware, an interface portionto intermediate between the middleware and the system LSI, an interfaceportion to intermediate between the middleware and the necessaryhardware other than the system LSI, and a user interface portion, andwhen integrating these elements to form the playback apparatus,particular functions are provided by operating the respective elementsin tandem.

Appropriately defining the interface corresponding to the middleware andthe interface for the middleware and the system LSI enables parallel,independent development of the user interface portion, the middlewareportion, and the system LSI portion of the playback apparatusrespectively, and enables more efficient development. Note that thereare various ways of dividing up the respective interface portions. Forexample, when the described video recorder 5 a, video recorder 5 b,audio decoder 9, and composite unit 15 are included on a chip in thesystem LSI 102, development of an interface portion between themiddleware to control these units and the middleware corresponding tothese units is performed when developing the chip. After completion,including the developed middleware and interface portion in a storageunit such as a memory of the playback apparatus, along with integratingthe chip into the playback apparatus, enables performing development ofthe playback apparatus and the chip in parallel, thereby improvingdevelopment efficiency.

Versatility is improved when the same interface portion is usedregardless of the type of developed chip and the middleware pertainingto the developed chip.

Needless to say, the portion structured as a system LSI in the abovedescription is not limited to being structured as an LSI, and mayinstead be configured with use of a signal processing circuit thatincludes corresponding functions to those to be included in the systemLSI.

INDUSTRIAL APPLICABILITY

The present application relates to a technique of compositing subtitlesand graphics with a stereoscopic video stream, and displaying theresultant data in the playback apparatus that plays back thestereoscopic video stream, and is particularly applicable in astereoscopic video playback apparatus that composites the subtitles andthe graphics as well as the stereoscopic video stream, and outputs theresultant data stereoscopically.

DESCRIPTION OF REFERENCE NUMERALS

-   100 BD-ROM-   200 playback apparatus-   300 remote control-   400 TV-   500 crystal liquid shutter glasses-   101 front end unit-   102 system LSI-   103 memory device-   104 back end unit-   105 nonvolatile memory-   106 host microcomputer-   107 network I/F-   110 BD drive-   111 local storage-   1, 2 read buffer-   3 virtual file system-   4 demultiplexer-   5 a, 5 b video decoder-   6 video plane-   7 a, 7 b image decoder-   7 c, 7 d image memory-   8 image plane-   9 audio decoder-   10 interactive graphics plane-   11 background plane-   12 playback state/setting register (PSR) set-   13 static scenario memory-   14 playback control engine-   15 composition unit-   16 HDMI transmission/reception unit-   17 left-right processing storage unit-   18 display function flag holding unit-   19 plane shift engine-   20 offset setting unit-   22 BD-J platform-   23 dynamic scenario memory-   24 mode management module-   25 HDMV module-   26 UO detection module-   27 a still image memory-   27 b still image decoder-   28 display mode setting initial display setting unit-   29 display mode storage unit

1. A playback apparatus that executes stereoscopic playback, theplayback apparatus comprising: a video decoder operable to decode avideo stream to obtain video frames; a video plane that holds the videoframes; a graphics plane that holds graphics data, the graphics datahaving a resolution of a predetermined number of horizontal and verticalpixels; a compositer operable to composite the graphics data within thegraphics plane with one of the video frames; and a shift engine operableto perform plane shifting of the graphics plane, wherein each of thevideo frames in the video plane is output as one of a right-view videoframe and a left-view video frame, the plane shifting of the graphicsplane includes: prior to compositing the left-view video frame by thecompositer, shifting each pixel of pixels of the graphics data withinthe graphics plane in one of a right direction and a left direction; andprior to compositing the right-view video frame by the compositer,shifting each pixel of the pixels of the graphics data within thegraphics plane in an opposite direction from a shift direction of thegraphics data prior to compositing the left-view video frame, and thentransmitting resulted shifted graphics data within the graphics plane tothe compositer for composition, the graphics data within the graphicsplane is for receiving a user operation, the user operation specifies adepth value, and a shift amount by which coordinates of the pixels ofthe graphics data are shifted is calculated based on the depth valuespecified by the user operation.
 2. A playback apparatus that executesstereoscopic playback, the playback apparatus comprising: a videodecoder operable to decode a video stream to obtain video frames; avideo plane that holds the video frames; a graphics plane that holdsgraphics data, the graphics data having a resolution of a predeterminednumber of horizontal and vertical pixels; a compositer operable tocomposite the graphics data within the graphics plane with one of thevideo frames; a shift engine operable to perform plane shifting of thegraphics plane; and an offset storage that temporarily stores an offsetspecified by an application, wherein each of the video frames in thevideo plane is output as one of a right-view video frame and a left-viewvideo frame, the plane shifting of the graphics plane includes: prior tocompositing the left-view video frame by the compositer, shifting eachpixel of pixels of the graphics data within the graphics plane in one ofa right direction and a left direction; and prior to compositing theright-view video frame by the compositer, shifting each pixel of thepixels of the graphics data within the graphics plane in an oppositedirection from a shift direction of the graphics data prior tocompositing the left-view video frame, and then transmitting resultedshifted graphics data within the graphics plane to the compositer forcomposition, the graphics data within the graphics plane is written bythe application, and a shift amount by which coordinates of the pixelsof the graphics data are shifted is calculated, at a time when an outputof both the left-view video frame and the right-view video frame for asame frame is complete, based on the offset temporarily stored in theoffset storage.
 3. A playback apparatus that executes stereoscopicplayback, the playback apparatus comprising: a video decoder operable todecode a video stream to obtain video frames; a video plane that holdsthe video frames; a graphics plane that holds graphics data, thegraphics data having a resolution of a predetermined number ofhorizontal and vertical pixels; a compositer operable to composite thegraphics data within the graphics plane with one of the video frames; ashift engine operable to perform plane shifting of the graphics plane;and an offset storage that temporarily stores an offset specified by auser operation, wherein each of the video frames in the video plane isoutput as one of a right-view video frame and a left-view video frame,the plane shifting of the graphics plane includes: prior to compositingthe left-view video frame by the compositer, shifting each pixel ofpixels of the graphics data within the graphics plane in one of a rightdirection and a left direction; and prior to compositing the right-viewvideo frame by the compositer, shifting each pixel of the pixels of thegraphics data within the graphics plane in an opposite direction from ashift direction of the graphics data prior to compositing the left-viewvideo frame, and then transmitting resulted shifted graphics data withinthe graphics plane to the compositer for composition, the graphics datawithin the graphics plane is for receiving the user operation, and ashift amount by which coordinates of the pixels of the graphics data areshifted is calculated, at a time when an output of both the left-viewvideo frame and the right-view video frame for a same frame is complete,based on the offset temporarily stored in the offset storage.
 4. Aplayback method for executing stereoscopic playback on a computer, theplayback method comprising: decoding a video stream to obtain videoframes; storing the video frames in a video plane on the computer;storing graphics data in a graphics plane on the computer, the graphicsdata having a resolution of a predetermined number of horizontal andvertical pixels; compositing the graphics data within the graphics planewith one of the video frames; and performing plane shifting of thegraphics plane, wherein each of the video frames in the video plane isoutput as one of a right-view video frame and a left-view video frame,the plane shifting of the graphics plane includes: prior to compositingthe left-view video frame in the compositing, shifting each pixel ofpixels of the graphics data within the graphics plane in one of a rightdirection and a left direction; and prior to compositing the right-viewvideo frame in the compositing, shifting each pixel of the pixels of thegraphics data within the graphics plane in an opposite direction from ashift direction of the graphics data prior to compositing the left-viewvideo frame, and then using resulting shifted graphics data within thegraphics plane in the compositing, the graphics data within the graphicsplane is for receiving a user operation, the user operation specifies adepth value, and a shift amount by which coordinates of the pixels ofthe graphics data are shifted is calculated based on the depth valuespecified by the user operation.
 5. A playback method for executingstereoscopic playback on a computer, the playback method comprising:decoding a video stream to obtain video frames; storing the video framesin a video plane on the computer; storing graphics data in a graphicsplane on the computer, the graphics data having a resolution of apredetermined number of horizontal and vertical pixels; compositing thegraphics data within the graphics plane with one of the video frames;performing plane shifting of the graphics plane; and temporarily storingan offset specified by an application in an offset storage of thecomputer, wherein each of the video frames in the video plane is outputas one of a right-view video frame and a left-view video frame, theplane shifting of the graphics plane includes: prior to compositing theleft-view video frame in the compositing, shifting each pixel of pixelsof the graphics data within the graphics plane in one of a rightdirection and a left direction; and prior to compositing the right-viewvideo frame in the compositing, shifting each pixel of the pixels of thegraphics data within the graphics plane in an opposite direction from ashift direction of the graphics data prior to compositing the left-viewvideo frame, and then using resulting shifted graphics data within thegraphics plane in the compositing, the graphics data within the graphicsplane is written by the application, and a shift amount by whichcoordinates of the pixels of the graphics data are shifted iscalculated, at a time when an output of both the left-view video frameand the right-view video frame for a same frame is complete, based onthe offset temporarily stored in the offset storage.
 6. A playbackmethod for executing stereoscopic playback on a computer, the playbackmethod comprising: decoding a video stream to obtain video frames;storing the video frames in a video plane on the computer; storinggraphics data in a graphics plane on the computer, the graphics datahaving a resolution of a predetermined number of horizontal and verticalpixels; compositing the graphics data within the graphics plane with oneof the video frames; performing plane shifting of the graphics plane;and temporarily storing an offset specified by a user application in anoffset storage of the computer, wherein each of the video frames in thevideo plane is output as one of a right-view video frame and a left-viewvideo frame, the plane shifting of the graphics plane includes: prior tocompositing the left-view video frame in the compositing, shifting eachpixel of pixels of the graphics data within the graphics plane in one ofa right direction and a left direction; and prior to compositing theright-view video frame in the compositing, shifting each pixel of thepixels of the graphics data within the graphics plane in an oppositedirection from a shift direction of the graphics data prior tocompositing the left-view video frame, and then using resulting shiftedgraphics data within the graphics plane in the compositing, the graphicsdata within the graphics plane is for receiving the user operation, anda shift amount by which coordinates of the pixels of the graphics dataare shifted is calculated, at a time when an output of both theleft-view video frame and the right-view video frame for a same frame iscomplete, based on the offset temporarily stored in the offset storage.